Great Way To Visualize and Communicate A/B testing, and to Make Good Use of “Big Data”

Pendlingskollen_Visualizing_A-B-Test_Using-BigData_MIkaelLeven

Visualization of A/B testing. Old traffic control system to the left, actual arrival time in the middle and the new predictive algorithm to the right. Currently the new algorithm has a distinct advantage with a score of 23,010 vs. 1,504 for the old system.

This is a great example that shows how we can benefit from “big data” to achieve a greater good, wether the information comes from historical data, is fed live through M2M sensors (machine-to-machine/Internet of Things) or a combination of the two.

In this example, StockholmsTåg use quite insipid information (at least when looking at it as isolated data) – the weight of a railway car – to make insightful predictions when there is a disturbance in the railway traffic. This is possible thanks to placing the insipid data into a broader perspective where the data comes to life and becomes meaningful. This way they are able to determine which railway wagons that contains the greatest number of people, so when there is obstacles on the tracks they can predict how this will effect the traffic in general and accordingly prioritize those wagons with most people, and by doing so they minimize the overall social effect of the inevitable (the obstacles itself is hard to avoid).

Pendlingskollen_Visualizing_A-B-Test_Scoring_MIkaelLeven

A/B test scoring. In this sample the new system has gained one point, illustrated by the green area.

And what is particularly interesting in this case is how they visualize their A/B testing, by giving the old and the new system a score based in their performance, and how they communicate this transparently directly to their end-users (the commuters).

StockholmsTåg has empowered their data to better be able to predict the effects of upcoming disturbances in the railway traffic, and they also found a great way of visualizing this. In the long run it also means they can have a more proactive approach towards their customers regarding disturbances and the potential workarounds, and thanks to this greater detail the commuters can make more well-founded decisions to make the most out of their time.

http://pendelprognos.technology/

Advertisements

Hands-on: Getting Started With Internet of Things and Home Automation

iot hubCurious about the Internet of Things and Home Automation and want to dig into it yourself, without knowing where to start?

Following my previous post, What is this (hot topic) ‘Internet of Things’?, an introduction to what Internet of Things (IoT) is all about, this guide will direct you through your first steps of how to get started with IoT and home automation. There is lot’s of great how-to tutorials out there covering various topics, products etc, and I have no attention to write yet another detailed tutorial on any of these subjects. However what I personally find missing is a great overview when you need to get started. And that’s what I aim to cover. What are the options? Which products do you need and what are the alternatives? Is there anything particular you need to know? What skills do you need? There is a lot of those questions that is hard to grasp when you are new to this broad and general subject.

So the goal with this guide is to assist you with your first steps of getting started with IoT, help you choose the appropriate path (for you) and to avoid typical pitfalls for beginners. I will also gather a bunch of useful links to various helpful resources and great tutorials.

I will take you through three important steps in the decision process and try to sort out what is what.

  1. Choose your project type: Of-the-shelve products vs. DIY projects
  2. Select the core technology: How will your units communicate with each other, which protocol/transmission method should they use
  3. Determine what hardware and software you need: Which components is needed and what are they used for
IoT_Home-Automation_Brand-Technology-Overview_MikaelLeven

An overview of some of the brands and technologies and their approximate relative positioning. This is by no means a exhaustive collection.

Step 1: Choose your type of project

One of the first things you need to do is to decide what your limitations are, how much time & money you intend to spend and what your end goal with the project is. Basically we can divide projects into any of these two following categories:

1) Consumer friendly with of-the-shelf products, and
2) Do It Yourself-projects with custom hardware builds.

Typically the first category does not require any programming skills, should you make do with the functionality provided out-of-the-box. This (normally) covers features like read temperatures, dim lights, schedule when to turn on/off lights and similar. Should your needs extends to more advanced features than that you will need to assume that (at least some) programming skills is necessary.

The choice of project type is primarily about simplicity vs complexity, time vs cost and limitations vs control/possibilities. It will be a tradeoff whichever you choose. But it is also possible to mix them as you like. Either you start with the of-the-shelf products and then switch over to the DIY-track, when you are ready. Or you let some parts of your system be of-the-shelf and other parts DIY. It’s entirely up to you.

Before going full throttle with a full Internet of Things project, I would recommend starting small and keeping it simple. That is, restrict your scope to only cover a single home automation set-up (to begin with), and keep the number of sensors/actuators to a minimum. And remember; it should be fun!

Consumer-Friendly Of-The-Shelf Products

Consumer friendly of-the-shelf products for home automation (

Consumer friendly of-the-shelf products for home automation (“smart homes”) and Internet of Things

This segment is all about choosing ready-made products which makes it easy to get started and achieve results. However as these are ready-made products, and targeted towards consumers, the price tends to get higher. And you will give up control (more or less) and the number of options will be restricted to what products each supplier and/or technology offers. The key aspects of of-the-shelves products is that they are limited in control and functionality, costs more but requires less time than DIY. And since they are consumer products you should expect quite good esthetics as well.

Do It Yourself (DIY)

RPi_DIY_Camera_MikaelLeven

DIY hardware for building your own smart home or Internet of Things-oriented device

DIY projects is where it gets really interesting and the possibilities are endless. Oftentimes you could get a somewhat lower cost by doing it yourself. And you have great control and full flexibility to customize to your specific needs. However, those benefits comes at a price; then time you need to spend and the requirement on your skills. I would personally say it’s more rewarding to do it yourself, but in the end it is your time and skills that sets the bar. DIY is about full control, lots of possibilities, lower cost, and with the tradeoff for more time consuming projects and higher complexity. And you will need programming skills, at least of some kind.

Step 2: Choose Your Communication Technology

IoT_Home-Automation_Brand-Technology-Overview_MikaelLeven

Overview of brands and manufacturers and their approximate relative positioning

Home automation and IoT is about devices communicating with each other. To enable this they need a mutual communication protocol. Within home automation these can typically be divided inte wired and wireless communication technologies. Wired is of course not as convenient as wireless while being (relatively) much cheaper, however they both have their pros and cons. So lets have a look at some of the most popular options.

Not interested in knowing how each technology works and why I came to the conclusion I did? Ok, then you could skip directly to the summary and conclusion.

Purely Wired Systems

1-Wire is a pure wired system that provides low-speed data signaling over long distances at low cost. Due to the requirement of physical wiring, this technology is (typically) disqualified in consumer-oriented home-automation set-ups. However as a signal bus between a gateway/micro-controller and severals sensors, this is a very cost effective solution. 1-Wire also have support for single wire earth return (requires only one actual physical wire).

Hybrid systems

X10 was initially wired, but today has support for wireless technology as well. X10 is an open protocol. When using the wired option (communication over the power lines) there is some limitations to take into consideration, something effectively making this option less practical.

KNX is a professional grade protocol with wireless support as well as multiple wired options (such as power line, TP cables etc). The fact that this is a professional grade protocol is reflected by the higher price level. For a large and advanced setup KNX offers a lot of possibilities.

Insteon is a proprietary system with both wired (power line) and wireless options. Insteon systems also have support for the X10 protocol which makes this a flexible option. The appliances are consumer-oriented and is of-the-shelves products. Uses mesh-networks.

Standardized Wireless Systems

ZigBee and Z-Wave is two very similar standardized and widely adopted wireless protocols with support for mesh network topology. Since ZigBee is an open standard the electronics can be provided by any supplier, while Z-Wave have their communication circuits produced by Signa Electronics. This means that ZigBee might have a slight advantage when it comes to costs thanks to some supplier competition, while Z-Wave might be slightly more expensive and offer better reliability and compatibility. At least theoretically.

Both ZigBee and Z-WAVE technologies is generally comparable to each other and offers roughly the same key features; wireless, energy efficient, long range (thanks to mesh topology they can cover distances of several hundreds of meters), easy-to-use, no requirement for a gateway* and with acknowledgement of sent commands. Each technology has several hundreds of companies backing them so there is no shortage of compatible units for either camp. Another benefit is the possibility to create your own units (using a Sigma transceiver circuit for Z-Wave or any circuit of your choice following the IEEE 802.15.4 specification, such as the Atmel “ZigBit”, specification for ZigBee). ZigBee can operate both on the unlicensed 2.4 GHz band (worldwide) or the 915 MHz (Americas and Australia) and 868 MHz (Europe) bands, while Z-Wave always operates at 868 Mhz. Generally speaking, a lower bandwidth increases the potential signal distance, giving 2.4 Ghz ZigBee (theoretically) inferior performance compared to the ~900Mhz ZigBee or the 868 Mhz Z-Wave. This would typically mean an indoor range of 10-20m for these devices and about 100m outdoors (theoretically up to 1500m in a very best case scenario with free line-of-sight, maximum power output, perfect conditions etc). Worth mentioning is the benefit of the mesh network topology that compensates for the limitation in effective range (simply explained; each device can forward the signal to the next device, creating a chain of devices where the limitation in range is between each two devices, which should give four units in a straight line indoors at least 30 m of effective range).

Z-Wave is supported by companies like Fibaro and D-Link, two companies with a clear consumer focus, as well as by other companies like Honeywell. ZigBee is supported by brands like Philips, LG and Belkin. Many companies additionally support both technologies, like Samsung, Verizon, Logitech, GE Appliances and Greenwave Systems etc. To learn more about the differences, check out this comparison between ZigBee, Z-Wave, Thread and WeMo.

* You can connect a switch and a light source without the need for a internet connected gateway, this however is quite limited and does not really falls within the home automation category.

Additional Wireless Systems

In addition to the standardized wireless protocols there is also other alternatives, proprietary systems operating in the 433mhz frequency band, like the System Nexa etc. Even tough being proprietary protocols there might be several manufacturers of compatible products, much like the Tellstick Net which can control devices based on System Nexa as well as systems from several other manufacturers. Many of these systems is generally cheaper than more qualified and flexible solutions like ZigBee and Z-wave. However for simpler set-ups, these consumer-oriented and affordable devices might be a good fit (you could pick up a starter kit with a Tellstick Net, three wall plugs and some additional appliances for about $150). These systems have an range of up to 30m (and significantly less with obstacles such as walls etc). There is products like Zipabox Duo that can control both these 433mhz-systems as well as Z-wave systems at the same time. One drawback with the 433 Mhz systems is normally the lack of two-way communication (which means the systems simply fires away a command and hopes it will be received, while the more advanced systems like Z-Wave/ZigBee actually acknowledges a successful transmission). This is something that might be important to consider depending on which type of system you intend to build (a light bulb that stays lit might not be that big of a deal, while a digital lock that remains locked when your kids stands in the cold freezing is whole other story, or when the door sensor missed en opening of the front door during the night and failed to activate your intrusion alarm). The 433Mhz line of products is all about easy-of-use and low cost. A typical use case for this technology is remote controlled lightning and wall sockets and it is within this area you will find the majority of the products.

Other Options

Some manufacturers, like Philips and D-Link, has chosen WiFi as the transmission method for their appliances. As this is a widely used and common technology it does not require any dedicated (and often costly) gateways. One significant drawback is the fact that WiFi is a quite power hungry and is not particularly cheap. And the greatest benefit of WiFi is the higher bandwidth, when compared to the other alternatives, however when building a network of small and cheap sensors and actuators there is seldom a need for any significant amounts of data to be sent over the network.

Bluetooth is a medium data rate and medium power consumption protocol. Not well suited in sensors networks as it requires paring with a master device (one-to-one). Also the higher power consumption makes it less suitable for sensor netowrks.

Bluetooth Low Energy (BLE) is a very low data rate and very low power protocol. These RF chips can run for years on an AA battery and so are used for things like heart rate sensors. BLE is supported by the latest mobile phones and laptops and so a phone or laptop could control the device, for example a LED lamp. BLE could also be used presence-awareness (for example using the iBeacon technology) which could be interesting in a home automation/IoT set-up.

For DIY projects you also have the option to build your own wireless communication system based on nRF24L01P (or any other cheap wireless transceiver) as well as creating your own multi-technology gateway (why not support Wifi, Z-Wave and ZigBee in the same box?).

Summary and Conclusion

Ok, so there is a lot of options (and the list above is by no means exhaustive), so we need to come to some sort of conclusion and narrow down the options.

I would rule out X10, even tough it is old and proven as well as having both wired and wireless support, mostly because it feels a bit outdated. KNX gets ruled out due to the unmotivated higher cost (and in some regards the lack of consumer focus). 1-Wire gets ruled out because it is solely wired (even though it still earns its place when it comes to short range connectivity between sensors and gateways, especially in DIY projects).

Personally I would look at either ZigBee or Z-Wave, if I was looking for a more capable and flexible system, or something similar to System Nexa (a low priced, simple and consumer oriented proprietary system in the 433Mhz frequence band) if I was looking for a simpler set-up. Insteon could also be an option, however it is neither as cheap as “System Nexa” nor as flexible and widely supported as ZigBee/Z-Wave (but might make sense if you fancy the one-supplier-has-it-all approach).

Since there is gateways supporting both 433Mhz and Z-Wave (like Zipabox Duo and Telldus Znet Lite) this would be a very good solution for beginners as you could start in a small scale. A first step would be to simply remote controlling some parts of your lightning using cheap 433Mhz products (like remote controls and wall sockets). Then you could add the gateway to make the set-up a bit smarter as well as adding the ability to control the devices from a smartphone (and now you can attach sensors, like a thermometer). Then, when you feel up for the task you can bring it to the next level and begin interconnecting each device with one another (make devices react on each other and possibly perform statistical analysis on the aggregated data). At this stage you could also bring more advanced Z-Wave products into the mix.

Step 3: Choosing Hardware/Products For Your Build

If you intend to stay with of-the-shelve products, and not even build your own gateway* or low-cost HTPC, you could skip directly to step 3b. Do you intend to build a DIY-project, or is generally interested in learning about the cheap single-board computers and micro-controllers, or intend to (at some stage) build your own gateway, NAS, HTPC, TOR Proxy etc, then the step 3a is for you.

* Even if you aim for a of-the-shelf project, you can in some cases significantly reduce the initial cost of the gateway by building your own single-board computer based gateway using relatively cheap hardware (say $90-100 instead of $300-550).

Step 3a: Singe-Board Computer vs Micro-Controller Unit

If you opt for a DIY project you need to choose a “maker platform” (hardware) to base your build upon. To simplify things a bit we could categorize these DIY maker platforms as either a A) single-board computer (SBC) or a B) micro-controller unit (MCU). In either case we are talking about quite small and powerful solutions, both of which are relatively cheap and power conservative. The actual choice of a single-board computer or a micro-controller unit depends on what type of project your are building and what your specific needs are.

Single-Board Computers

Raspberry_Pi_B+_rev_1.2.svg

Raspberry Pi single-board computer

If you need to run a full computer operating system (could be Android, Rasbian, Windows 10 or Ubuntu, it depends on the board) you should go for a single-board computer like the Raspberry Pi (or any of the many alternatives). The same goes if you need other computer-like functionality, such as; running a GUI, install custom software, need some significant processing power, or if you simply have the need for a specific programming language/platform (such as NodeJS). The downside with these boards is the cost and less options for GPIO. The upside is greater flexibility and more power, and quite frankly, you will get a hell of a lot for your money. Take the Orange Pi single-board computer, a beefed up Raspberry Pi, sports the following features; 1 GB of RAM, wifi, gigabit ethernet, quad core processor running at 1,6 Ghz, SATA-support etc. All this in packade small enough to fit in your palm, and with a cost of about $39. Personally, I think that is amazing! Best of all, these computers typically draw 5 watts of power (or less), something I find pretty darn good (as it would require twelve of these to even draw as much power as one typical lightbulb).

Most SBCs is not as powerful as the Orange PI. A typical limitation is the LAN which often is 10/100 Mbit. And in most cases there is only a SD card reader for storage, no S-ATA connectivity. And USB is often limited to a few ports, which only supports the USB 2.0 standard. Also WiFi is by no means attached by default (though not that expensive to add via USB). So you need to pay special attention to what your explicit connectivity needs is. Performance of the LAN would under normal circumstances not pose a problem, the lack of WiFi could on the other hand be a significant limitation. The same goes for the performance and number of physical USB ports. Adding a lot of peripherals requires a more powerful power unit for the SBC, and the CPU will be a bottleneck when trying to squeeze maximum performance out of the USB 2.0 standard.

A SBC is, kind-of, also a MCU. Like an MCU a SBC will normally have GPIO and much of the same abillities, just with more power and additional options. So a SBC would make a good simple development board, especially if you want to write your program/sketch in a specific language (Arduino “sketches” is written i C, while SBC’s typically can run any programming language you want). There are good libraries for Python, LUA and NodeJS, and even .NET/C#.

Micro-Controller Units

Arduino Uno micro-controller

The micro-controllers could be divided into two sub-groups. First we have the standard classic Arduino-type medium sized development boards, which have a bunch of GPIOS (inputs and outputs, I will come back to that later) and with support for common “shields” (add-ons) that extends the base platform with additional functionality such as PWM controller, Wifi, ethernet, SD card reader etc.

Trinket micro-controller

Then we have the smaller sized (“tiny”) development boards (which often have very limited number of GPIO’s and comes with a significantly smaller package and lowered power consumption).

There is also various other models of these MCU developments boards (such as the “mega” with far more GPIO pins), however in most cases either the standard sized Aurduino or the “tinys” would do just fine for your project. Also worth mentioning is the many brands of Arduino boards (thanks to open-source hardware), as well as the phenomenon with Ardunio-compatible boards (not actually Ardunio per see, but compatible with Ardunino shields). Additionally there is development boards that is not really compatible with Ardunio itself (such as with the shields), while still being compatible with the Ardunio IDE (the development environment). Neither of these options is crucial for this guide so we leave them out to keep things simple.

When single-board computers more or less are full-fledged computers (albeit not as powerful as your everyday laptop), the micro-controllers on the other hand typically are small dedicated devices with a specific purpose of controlling physically connected hardware. Instead of running an operating system and additional software, MCU’s are equipped with a small optimized “bootloader” (think of it as an micro-OS) and a single dedicated program (think of it as a command/script on your computer). For Arduino these programs are called “sketches” and are programmed through a PC via USB using the Arduino IDE (which can be used to program any Arduino-compatible MCU).

One common limitation is the lack data communication capabilities (such as wifi and LAN), which effectively makes these devices unable to single-handed communicate over a local network or the internet, unless they are extended with a network “shield” (LAN or wifi). It would also be possible to connect to a host PC to add networking capabilities, however this would probably take away most of the benefits of these low-cost low-power devices. Although MCUs typically are dependent on a host computer for programming, there is some extra powerful models with the possibility to “program over the air” thanks to on-board network capabilities, such as the Arduino Yun (which basically is a standard Arduino with an built-in wifi-module/”shield”), the Arduino Zero (with LAN) and the Adafruit Huzzah (which is a very tiny, cheap and capable wifi micro-controller). MCU’s also lack any real processing power. This makes them suitable for reacting to sensors and controlling other hardware, but not suitable to analyze data.

There is also additional wireless communication circuits like the NRF24L01 (custom wireless communication) or generic wifi-circuits that could be attached to any MCU to add network capabilities, though we leave them out-of-scope at this stage for simplicities sake.

Most MCU’s are programmed using C (Arduino “sketches”), while there is other options (such as the very convenient WiFi-enabled tiny MCU ESP8266 with LUA support through NodeMCU).

Shared Feature – GPIO

Common features between both single-board computers and micro-controller units is the GPIO (General Purpose Input Output) pins. Basically what GPIO is about is reading or controlling the physical state of a pin on the board. Digitally this could be either switching a LED diode on or off (output), or check wether a button is triggered or not (input). On analogue ports this could be reading the intensity of resistance from a thermal sensor (input), or adjusting the intensity of a light bulb (output). Each pin can often be configured as either input or output, but never both at the same time.

There is also more advanced techniques for digital IO, such as; PWM (Pulse Width Modulation) which is a way to switch on and of a signal multiple times in a short time and thus be able to control the perceived intensity (think of it as a “dimmer”) of a LED or to control the RPM of a engine, I2C which is a serial bus interface enabling multiple units/sensors to connect using the same shared wires (similar to 1-Wire, with I2C having greater speed and requires additional wires), to even more advanced digital interfaces like UART and SPI (different to I2C in being full-duplex/two-way synchronous serial communication, typically in a master-slave arrangement).

One common key difference between SBCs and MCUs is that SBCs (typically) only support digital input/output while MCUs tend to support both analogue and digital input/output. MCUs also typically have a greater focus on input/output and therefore have a greater number of GPIO pins.

A Short Recap

Examples of

Examples of “maker” and IoT hardware

Single-board computers: when you need an full OS, a GUI or other computer-like functionality. Suitable for building a NAS, a HTPC, a small web browsing PC, a Tor-proxy or why not building your own internet connected IoT gateway.

Micro-controller unit: when you need a cheap, power efficient solution w/o OS and significant computing power. Suitable for most hardware-builds like robots, home automation etc.

Offline MCU: when you can keep the board connected to a PC at all times, or when you do not need to send data back to another device. Suitable for building a automatic plant watering system or a robot.

Online MCU: when you need to send data back to another device/network or when you want the convenience to be able to program the board through a network (w/o the need for a direct USB PC connection). Suitable for building a remote unit reporting sensor data to a database/IoT-framework.

Step 3b: The Products You Need

What You Need For a Of-The-Shelve Type Project

Choosing the appropriate components for your of-the-shelf system:

In every home automation system you will need some sort of gateway, the “heart” of the system. Depending on technology this could be products like Fibaro Home Center 2, Zipato Zipabox, Tellstick Net, Digi XBee or Nexa Gateway.

Then you needs some sensors and/or actuators. This could be temperature sensors, water moisture sensor, wall outlets, branching outlets, dimmers, pumps, engine/stepper motors etc.

Finally you will probably want some sort of software/application to control your devices. In simple scenarios this would be an app to control your lights, but it could also bee considerably more advanced than that. When you go for an of-the-shelve set-up the manufacturer typically provide you with the software needed, and this should be sufficient for your first project (many even provide the ability to control devices through your smartphone).

What You Need For a DIY Type Project

IoT-Do-It-Yourself-approach-DIY_MikaelLeven

You will need some sort of gateway, the “heart” of your system. The gateway will have two responsibilities; 1) send data back and forth to the internet, and 2) communicate with your sensors and actuators.

You will probably want to base this gateway on a single-board computer, such as the Raspberry Pi (check out this buyers guide to choose your first “Pi”). Depending on your chosen communication technology you will need to extend the Raspberry Pi with additional hardware (could be wifi USB dongle, RazBerry or XBee for example)

TheThingbox_NodeRED_Programming_GPIO_MikaelLeven

“Visual programming” in Node-RED

Node-RED, which is based on NodeJS, is a great way to get started with Raspberry Pi as it lets you “visually code” your solutions. (The Raspberry Pi will support Python out-of-the-box if you prefer this, but then you will have to do without Node-RED). And to make things even easier, there is a ready made image called The Thingbox, which is great for beginners as it is a complete installation of Rasbian (the OS for the Raspberry Pi) along with NodeJS and Node-RED. Everything you need to get started (software wise) as simple as download, install and you are done.

Ubidots_Control_IoT_MikaelLeven

Easy-to-use web based Ubidots

Even tough not necessary, I would also recommend connecting things to an online IoT framework as it will make it a lot easier to control eg. using your smartphone. Here we have Ubidots which is really simple to use (and is supported in Node-RED). And like icing on the cake; there is a pedagogical and simple-to-follow tutorial on how to combine Raspberry Pi, Node-RED and Ubidots. Just follow a few steps and your are done. It can’t be much easier than that.

So, what you need up to this point is basically only:

  1. A Raspberry Pi
  2. The Thingbox image
  3. An Ubidots account
  4. To follow the Ubidots + Node-RED tutorial

Then you need some sensors and/or actuators. This could be temperature sensors, water moisture sensors, relays, wall outlets, branching outlets, dimmers, LED’s, pumps, engine/stepper motors etc. This could be either of-the-shelf products based on Z-wave or ZigBee, or stand-alone hardware sensors like DHT22 Temperature and Humidity Sensor as well as simple buttons and LEDs.

Finally you will probably want some sort of software/application to control your devices. As we already have Ubidots in our setup you already have some means of controlling your devices. And with their API it is easy to extend with your own custom software to control and or visualize your devices data.

In addition to the Thingbox-solution (with Node-RED and Ubidots), there is lots of other great options. Python is natively supported by Raspberry Pi out-of-the-box, for NodeJS there is many great libraries for simple GPIO control, for netDuino you have a .NET library, and the Jonny-Five is a great NodeJS library which supports multiple development boards like Raspberry Pi, Arduino many more.

Next Step: Choosing Sensors & Actuators and How to Begin With Hardware Development

Now that you have have decided on the core parts of your system it is time to getting into the real action. That is start getting values from sensors and make things happen at (or around) your home. The next post will show a birds eye view on the different type of sensors and actuators that you could use and what they could be used for.

Additionally a brief introduction to cover the basics of hardware development will be part of the next post (for those interested in general, or specifically opted for a DIY project). Like in this post I will not get into the details how to actually do these projects, but rather point you into the right direction to get started as well as to mention some typical pitfalls for beginners.

Inspiration – To Give You an Idea How to Get Started

In the meantime, the following is some inspiration on different projects you could get your hands on if interested in hardware development.

Raspberry Pi Hacks

12 Raspberry Pi projects: http://www.alphr.com/raspberry-pi/raspberry-pi-2/1000043/best-raspberry-pi-2-projects-14-cool-things-microcomputer
RetroPie nintendo clone: http://1.cdn.eprofits.com/AHCV4wwjEBW8gH0CHBDzpeVQwjhS9MP6F0qnaZHKby2w1j7yJ63s7W_1TJ_hz2gCblZbsfuc55c-yKh-eknicg/1417-the-coolest-diy-projects-of-the-google-raspberry-pi-community-1.jpg
A robot (nuff said): http://latestcomputergadgets.com/wp-content/uploads/2015/03/raspberry_pi_robot.jpg
Good looking case: http://i.kinja-img.com/gawker-media/image/upload/s–Sq2EsLGj–/c_fit,fl_progressive,q_80,w_636/714162336531572655.jpg
Handheld Raspberry Pi: http://www.bitrebels.com/wp-content/uploads/2012/12/raspberry-pi-laptop-assembly-1.jpg
Onion Pi (Tor proxy): https://www.raspberrypi.org/wp-content/uploads/2013/06/onionpi.jpg
Altoids tin computer: http://cdn.instructables.com/FLY/713V/HAQ33LE0/FLY713VHAQ33LE0.MEDIUM.jpg
Raspbery Pi Cluster: http://cmg.soton.ac.uk/static/images/news/2012raspberry/raspberry-soton.jpg

Arduino hacks

A robot: http://i.ytimg.com/vi/zjCoQ5W0wE0/maxresdefault.jpg
Another type of robot: http://cdn.instructables.com/FDW/NYYO/HQVLDKEI/FDWNYYOHQVLDKEI.LARGE.jpg
Quad copter (“drone”): http://www.engblaze.com/wp/wp-content/uploads/diy-drones-quadcopter.jpg
Flamethrowing(!) Jack-O’-Lantern: http://www.instructables.com/id/Flamethrowing-Jack-O-Lantern/
Automated greenhouse: http://www.instructables.com/id/Backyard-Automated-Greenhouse/
DIY home security system: http://i.ytimg.com/vi/dRCnccv_dVE/maxresdefault.jpg
20 cool Ardunio projects: http://www.instructables.com/id/20-Unbelievable-Arduino-Projects/

UPDATED:
Added a reference to Your First Raspberry Pi: A Buyer’s Guide.

What is this (hot topic) ‘Internet of Things’?

InternetOfThingsAlmost every day topics such as Internet of Things, wearables, smart watches and home automation is mentioned in media and other sources. At least on remotely tech-oriented sources. Also mentioned in these contexts is considerations (or rather concerns) about security and privacy. These topics is close to my heart and I wanted to give my point of view on some of them, while also giving a brief introduction into the (mysterious?) area Internet of Things for those not really sure what this is. Kind of giving you a quick-start guide to encourage further reading. This post turns to both those without prior knowledge but with an general interest to learn what IoT is, as well as those already familiar with the topic who are ready to get started hands-on with IoT but want learn before they start. The intention is to follow up this general post with more detailed post digging into the technologies and how to get started when you actually begin building the first Internet of (Your) Things.

So what is this “Internet of Things”?

It is really hard to accurately define what Internet of Things (IoT) is using one short description. In general IoT is a concept where “anything” is connected to the Internet, or rather “anything connected to anything” (which is more true to the basic idea of IoT). We could called this “network of everything”. From my point of view, the best way to understand what IoT is actually is to talk about what it could be used for. The greater mening of the concept.

Automation (“smart homes” and industry automation) have actually been around for quite a long time. Historically (home) automation has been about enthusiasts creating nifty solutions for their private homes, mostly because it is fun but also to make their lives somewhat easier. It could be wirelessly controlled lightbulbs or automatic plant watering systems. Likewise the manufacturing industry has a lot of automation and monitoring going on. And aerospace research is also very much about automation and monitoring. One thing in common, from a historic point of view, is that these technologies where costly, inaccessible and often required some expert skills in electronics. And not to mention by far not as powerful as they are today.

The last decade this has truly changed. Today there is a lot of different, and cheap, solutions available to consumers. We have access to a extensive set of of-the-shelf consumer-friendly products to create smart homes. Additionally we have access to great amounts of processing power. All these devices getting cheaper, smaller and more energy efficient. And it is here we are getting closer to the true Internet of Things.

Basically IoT is about connected devices and sensors. Not necessarily internet-connected, but at least, connected devices. They could be connected to some local gateway/hub at your home gathering all data from these sensors or they could be connected to the Internet and the cloud. The benefit of IoT is when you have multiple devices connected to a common place where this data is aggregated and can be used to add new areas of use or to increase smartness of existing functionality. A temperature sensor is still just a mere temperature sensor until you pair it with one or more additional devices that can act on the sensor information to achieve new additional and/or smarter functionality (think of it as 1+1=3).

Internet of Things Simplified ArchitectureLet’s get a bit more technical

We could, generally speaking, look at IoT architecture using this simplified model with four layers/tiers*.

  1. Sensors and devices (collects data)
    This is where all information (data) is originated and created/obtained
  2. Hubs/gateways (proxies/sends data)
    In the web of interconnected devices, hubs is responsible for gather data from (possibly) multiple devices/sensors and distribute them throughout the network. Typically this layer is when data goes from being on-premise (local) and leave to the Internet/cloud (global).
  3. IoT software platforms/frameworks (stores and analyzes data)
    This is the very core of the IoT architecture where all data from all sensors and devices, via hubs/gateways, are gathered and stored. Typically this is the place where data being analyzed and some event could be triggered.
  4. End-user applications (visualizes and reacts on data)
    The fourth layer is where data actually becomes useful. I.e. this is when data either being presented/visualized or when data triggered an event that actually leads to something actually happens (a change of state of any kind).

* Each of these tiers could actually be combined into one and the same product/solution. It would be absolutely possible, and viable, to combine multiple sensors and one hub into one physical product. Or to combine tier 3 & 4 to create a service that both stores, analyzes and visualizes the data. Or in a particular situation you might want to have a local micro IoT at home where you have combined at least tiers 2 through 4 into one product and only have detached sensors. And all of this is only a simplified model with some examples as there is no definite rule exactly what IoT is and how is to be used.

Neither of these layer/tiers is technology or manufacturer specific. However between each tier there is a need for a common language or a contract for information exchange. It would be possible for a hub to use one language to receive data from sensors and another language to send data to a IoT framework. But the sensor cannot use different languages.

This general concept would work equally well with proprietary protocols and products as well as with open source products and standardized protocols. And it would work through Internet, a typical local computer network or any non-standardized “private network”. Both wired and wirelessly. Internet of Things is not a technique but rather a concept.

The BFF of IoT

IoT also have a best friend, or sibling, which is one key part when we move from simple automation to something really smart. Using “big data” – i.e. lots of processing power with access to huge amounts of data – we can make truly smart technology using predictions and analyzing of behavior patterns. Without “big data” IoT would be nothing more than a lot of connected devices without any greater meaning. Like a thousand people standing still in the same place, no one interacting with one another. That is lots of people, but quite meaningless as they don’t do anything. Compare this with these 1,000 people actually interacting with each other, exchanging experiences and together forming patterns. Think of this combined source of knowledge with one thousand brains in one place communicating with one another. This is IoT + “big data” – when this “network of everything” comes to life and becomes meaningful.

What is I(o)T good for and why should you and I use it?

A common (mis)perception of IoT is that every home appliance and device should be connected to the internet. However, IoT is not really about 50 million internet-connected fridges, but rather about connected sensors and devices where it makes sense. Sure, it would be cool if the fridge could order some milk when the milk is almost depleted. It’s arguable if that’s really practical. Maybe you want to wait just a little bit to order more stuff, or possibly you are about to go on vacation (and then you wouldn’t want milk in the fridge getting old).

Looking at a more practical approach to the fridge scenario could be: each package of milk could have a smart label armed with a sensor monitoring the state of the milk and a short range wireless communication unit (RFID/NFC). When the milk is nearing it’s end date the sensor sends a wireless signal to the fridge, which in turn could display a alert on the fridge’s display (or, by all means, it could also send a tweet or alert of some kind to your phone as well). Additionally your “remember the milk”-app (grocery shopping list) on your phone could have “one package of milk” listed as a suggested purchase. This could be both a viable and practical example scenario where IoT could enrich or simplify our lives.

To give a better idea on the multitude of scenarios where IoT in one way or another could be used I have collected some examples below. This is by no means a exhaustive list of all the possible scenarios. We should also expect many new ways to utilize the IoT concept that we cannot think of (or imagine) today. Much like we couldn’t predict what impact smartphones would have and how we would actually use them today.

  • Green tech – Smart technology to predict behavioral patterns to save energy consumption and use resources smarter. For example if we have smart heating systems that will report to energy suppliers about upcoming expected usage levels, time and
  • Home automation – Devices that make our traditionally “dumb” houses, smart. Mostly a matter of comfort.
  • Safety – Using various devices interconnected we could for example rest assured our kids have come home safely from school (and follow their way home). With connected locks the house could automatically get unlocked without the need to hand out physical keys to our kids or require the to remember pin codes.
  • Surveillance – Internet connected surveillance system monitoring your home, office or a public place like the subway.
  • Predict (global) health trends – Using wearables and behavioral patterns, maybe the combination of drugs for a particular illness sold, could give us quick indication of early stage epidemics
  • Optimized transportation – See patterns in how humans (or drones) travel and optimize the route accordingly
  • Proximity based interaction – based on your location and proximity relative to other things, this could be used as means of interaction and identification (or why not authorization of payments?)
  • Personal health monitoring – Map activity patterns and personal health status

There is also some examples of typical places where these devices and sensors could be used:

  • At home
  • In a vehicle
  • In an industry
  • On, or in, your body – aka “wearables”

A real world (hypothetical) scenario

To further clarify the benefit of IoT we could consider a more concrete scenario. Let’s begin with you, your home and a simple heat source (really doesn’t matter which type, but let us assume it is a electric radiator used to warm your house).

In the first stage, pre-IoT, your heater is simply on or off. When you set it to “on” it simply uses 100% of it’s maximum energi usage, regardless of how warm or cold your room actually is, and you have to manually control it.

Let us now take this to stage 2, the first step to a smart home. We add a temperature sensor with the ability to control your radiator – On or Off. This is what typically is called a thermostat. Now we have a basic micro IoT network with two (or actually three) devices. One input device (the sensor – thermometer), a hub (the thermostat with controller logic) and the output device (the heat radiator). With this basic setup we have somewhat intelligent heating that keeps our room heated without excessive usage of resources (electricity). However it is not really smart yet.

In the next stage, where we moving towards a more true IoT solution, we also add your telephones GPS to the mix. If the GPS tells that you are far from home the thermostat will lower the temperature with 2°C. And when the GPS reports you are getting closer than 2.5 miles from home it will begin heating back 2°C. Thanks to this smart approach you always have a perfectly tempered home, while at the same time have lowered your overall energy consumption (since there is no need to heat the room when you are not there). Already with as little as two sensors (GPS + thermometer) and one output device (heat radiator) connected to an IoT hub you have achieved a smart home with a smaller electricity bill.

We could of course stop there and be satisified. However let us also add another house to the mix, 5 miles from your home. This house also has an IoT gateway as well as an outdoor temperature sensor. Using the data from this house, with triggers on increasing/decreasing outdoor temperatures, we could predict the needs for heating at our house. So when the temperature drops at the other house we could begin increasing the heating in our house, and as our house hasn’t already being cooled down due to the lower outdoor temperature, we could run the radiator at a lower level which consumes less energi (i.e. more energy efficient). Another benefit is that we would attain a more stable indoor temperature as we can begin the heating before the temperature drops outside and thus avoid larger variations. Now we have a truly smart home which is even more energy efficient and also more comfortable.

At this stage we have achieved benefits mostly from a personal perspective, we have yet to achieve any greater benefit on a larger scale. Sure, our lowered electricity bill does contribute to an globally decreased need for energy which is environmentally good. Should we add another party to this mix – the power companies – we could attain even greater effects. Let’s consider if our, now smart home, also could communicate with the systems of power companies and let them know how our heating equipment currently operates, and also which changes our system has predicted. For example if our house is running at max levels and our neighbors house needs heating, the power company could tell our neighbors’ system to wait for a short while before starting. This is possible thanks to our house has reported that the desired heat level soon is achieved. Such intelligent system achieves two important things for power companies. First, their maximum energy supply level could be lower, secondly they can predict when they actually will need the power and thus minimizes the need to have overcapacity to manage unexpected peaks. The exact same pattern is equally applicable for washing machines, dishwashers etc. Looking at this at a grand global scale this could have a significant impact on global energy consumption and by extension the environment. And this is one reasons why IoT is so promising and interesting.

Security and privacy considerations

Wether you are a business creating your own products (alternatively an independent maker) or a consumer, you need to factor in aspects like security and privacy. Below I will shortly describe what I think these two aspects mean in the context of Internet of (My) Things.

Security

The security part is primarily about technical, digital and physical control of access. Every step of the chain must be as secure as possible. There will surely be a tradeoff between convenience and security and here you need to value how much convenience you are prepared to let go in favor of security. Or how much security you are willing to offer to gain convenience. From a consumer point of view this would probably be in the are of how you connect sensors/devices to the other parts of the system. And from a business side of things this would surely be related to how mature your audience is and thus how complexity they can cope with when handling your products. In either case security is paramount, both for individuals but to the market in general (should IoT and cloud based solutions be considered unsafe and unreliable the market will surely notice a significant setback). Security first, convenience second.

Privacy

In contrast to security, integrity isn’t that much about physical or technical limitations, but rather about how your information could be accessed and (mis)used. From a business point of view this is very much about anonymizing data before making it publicly available and to have a clear policy about storage and handling of end-users data. Looking at it from a consumer perspective this is mostly about choosing between lokal solutions (where you have full control) or opting for a cloud based solution. In the latter case the comnsideraton would be to choose a supplier who you trust and you find to have a authentic data integrity policy. And one last word of caution: without security there is no integrity. The first step is to secure the data and communication.

IMG-CAPA-IOTConclusion

What IoT really is about: gather lots of (sensor) data, process and analyze these large quantities of data, make predictions and finally make good things using these insights. Especially when we monitor and analyze data over sustained periods of times we can draw conclusions on behavior and be really smart and predictive when taking action.

Hope this gave you an basic idea what IoT is and the purpose/benefit of it. And please let me know what you think. Maybe there is something missing in this overview. Or it might be some part that needs further clarification. In any case I would appreciate the feedback.

Explanation of terms

Some new words, but what do they mean?

  • Internet of Things – A network of devices/sensors connected to each other
  • Big data – Large amounts of processing power analyzing biq quantities of data, and more importantly, the ability to do this i real time to instantly trigger an reaction based on the analysis.
  • Hub/gateway (in this context) – A sort of computer resposnible for send information between sensors and the IoT software/frameworks. This could also be called a proxy or a “middle man”.
  • Sensor – A (typically) small device with some sort of measuring/probing capabilities
  • Device (in this context) – Any form of electrical device that might be connected to the IoT network to supply the network with data (such as smartphones, wearables or computers in general). The difference to sensors is that these devices does not really measure anything, but rather supply data in other ways.
  • Wearables – small devices placed at, or within near proximity of, your body. Apple Watch for example.

Next step: hands-on with IoT technology

Keep an eye out for the next post in this series if you are interested in getting your hands dirty with IoT technology yourself.