The Internet Of Things
An ever growing number of devices is added to the things on the internet (IOT).
Some of these things are featured on my web site, and more of them will be featured here in the future I'm sure.
But how can you make all these things interact with each other in an organized and flexible way?
One of these ways is via the MQTT protocol.
It is a light weight publish/subscribe protocol.
Sensors (publisher nodes) publish their data to the MQTT server, which is called a broker.
This broker then sends these message to every actuator (subscriber nodes) which are interested in that data.
In short an MQTT broker simply relays sensor data to the actuators.
Therefore you don't have to program a sensor to talk to particular actuators in order to get the desired results.
MQTT allows you to make very flexible and powerful IOT applications.
In the example above you can create several applications, with just 3 sensors and 3 actuators.
For instance, when it is too cold and you are in or nearing your house, the central heater can be switched on.
When you're leaving, or when you're supposed to be in bed, the central heater can be set to a lower value, saving energy.
When it is morning and you're at home it might start the coffee machine for you.
But it might skip making coffee in the morning automatically if it detects that you're not home.
In the winter months the system can also switch your lights on and off at preset intervals or conditions.
The particular MQTT implementation we're going to use here is Mosquitto, which is a widely used and supported Open Source MQTT broker.
Installing and using Mosquitto is easy enough. It is so straight forward that it is hardly worth writing about it. However, unless you want to constrict your setup to your own private network, I should strongly urge you to add some security to the basic setup. Security in most IOT solutions usually leaves a lot to be desired. That's why I want to do it differently, which justifies writing about it.
Installing Mosquitto On A Raspberry Pi
Installing the Mosquitto broker on a Raspberry Pi, or on any other modern Debian based system, is quite easy. First ensure that your system is up to date, then run the following command.
sudo apt-get install mosquitto
Now you'll also need client software which can connect to our newly installed broker to test its operation. This can be done on the server machine itself, or on any other machine in your local network. Even on Windows machines, yikes. On a Raspberry Pi type:
sudo apt-get install mosquitto-clients
Now open two terminal screens, or use the screen or tmux commands if you know how to use them. In one of the terminals type the command
mosquitto_sub -h localhost -t test
and in the other terminal type the command
mosquitto_pub -h localhost -t test -m "hello world"
and you'll see the message "hello world" appear on the first terminal.
What has just happened?
The -h option will tell both commands what host should be used.
With host we mean the broker or server.
This can be localhost, if the client runs on the same machine as the server.
Or it can be any other computer's IP address in your local network.
The first command subscribes to a topic, indicated by the -t option.
In our example we subscribe to topic test.
And then the program waits for someone to publish something to the topic test.
You can use any topic you like, or even an hierarchical topic like for instance sensors/temperature/livingroom.
If a topic doesn't exist yet on the broker it is simply created for you.
The second command publishes something to the test topic.
In this case we publish the message "hello world", started by the -m option.
This message is then automatically sent to the other screen because it has subscribed to the test topic.
You can have multiple clients publish to one or more subscribers, which makes the system extremely flexible.
Type Ctrl-C to stop the mosquitto_sub program and return to the command prompt.
Just as an encore: Run the following command in case you want to use MQTT with Python or Python3:
sudo apt-get pip install paho-mqtt
Are We There Yet?
See, installing Mosquitto is easy enough.
And it works, right out of the box, on a simple Raspberry Pi.
Now you can connect all your DIY nodes to your very own MQTT broker.
It's a good way to familiarize yourself with the publish/subscribe methods of MQTT before you need something bigger.
There is only one limitation though, your MQTT broker is bound to your own private network only.
You can't connect your mobile phone to your MQTT broker when you're on the move for instance.
So you can't use features like world wide remote controlling your IOT gadgets, or use OwnTracks to start your central heating system when you are approaching your house for instance.
Believe me, with the out-of-the-box installation of Mosquitto it is a good thing that you can't connect to it over the entire internet yet. Not only you, but anyone could connect to your Mosquitto broker. They can abuse your broker, or even flood your IOT devices with false and incorrect data, or switch on the stereo at full volume in the middle of the night.
So let's take it a step further and add some security so it will be safe to allow your broker to be connected to the internet.
Adding User Name And Password
The first step is to add a user name and password to your Mosquitto broker. That way a node (client) has to authenticate itself before it can be allowed to access your broker, which should keep the bad guys out of your system.
First create or edit the file /etc/mosquitto/conf.d/config.conf. We're going to edit this file a few times for other purposes. I'll include the final version of this file at the end of this page for your convenience. Then add the following text to this file:
allow_anonymous false password_file /etc/mosquitto/passwd
Now you can create the password file and add the first user to it:
sudo mosquitto_passwd -c /etc/mosquitto/passwd username
You can chose any user name you like of course. You're asked to enter a password for this new user twice. Subsequent users can be added to the password file with the following command:
sudo mosquitto_passwd /etc/mosquitto/passwd otheruser
Deleting a user from the password file is done by the next command:
sudo mosquitto_passwd -D /etc/mosquitto/passwd olduser
We have made some configuration changes, so we need to reload them into the mosquitto daemon. Please note that you'll also have to restart the daemon when you have made any changes to the username and password settings.
sudo service mosquitto restart
After this change our previous test example won't work anymore because the authentication fails now. So in order to test the operation again you'll have to add the user name and password to the commands in both terminals:
mosquitto_sub -h localhost -t test -u username -P "secret"
mosquitto_pub -h localhost -t test -m "hello world" -u username -P "secret"
I don't think that was too complicated either.
However, it is still not very safe to use this setup over the evil internet.
Passwords are sent as clear text, which is never a good idea.
Granted there are still loads of web hosters who allow their users to upload their files over FTP, which also sends passwords as clear text.
But because others are neglecting the safety rules doesn't mean we'll have to ignore them too.
So let's encrypt our connections, so no one can eavesdrop on us before we are going to expose our MQTT broker to the internet.
Setting Up Encryption
You'll Need To Have A Domain Name First
Before you can setup encryption you'll have to assign a (sub)domain name to your Mosquitto broker machine.
If you don't have a domain name you can set one up for free at Freenom.
The domain name should of course point to the public IP address of your Mosquitto machine.
If you don't know what your current public IP address is you can look here
.
Be sure to read the Network issues section below if your IP address changes regularly when your ISP is stupid enough to use dynamic IP addresses.
It may take a while for a new domain name entry to propagate over the internet.
So the next steps might fail initially if they are executed too soon.
Simply wait a bit and try again.
Get Some Additional Software
We're going to use a free SSL certificate from Let's Encrypt. These certificates are free, but only valid for 90 days. Fortunately renewal of the certificates can be fully automated using their certbot software.
Unfortunately certbot is not included in the Debian Jessie's repository.
Therefore things may get a bit more complicated than absolutely necessary, depending on what system you are installing it on.
Your network topology may also be different, making it a bit more complicated to create a universal tutorial.
This means that sometimes I describe some steps which may not be necessary for you, or the steps might be a bit different for you, depending on your network situation.
So let's start by installing certbot first:
echo 'deb https://ftp.debian.org/debian jessie-backports main' | sudo tee /etc/apt/sources.list.d/backports.list
The step above adds the jessie backports repository to your Raspbian Jessie repository. However it may or may not be necessary for you. For instance it is not necessary if you're running Debian or Raspbian Stretch. The easiest way to find out whether it is necessary is to skip it at first and when the next steps fail you may execute it before you repeat the steps below.
sudo apt-get update
Now you may get the error below:
W: GPG error: https://ftp.debian.org jessie-backports InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 8B48AD6246925553 NO_PUBKEY 7638D0442B90D010
If you do, execute the following commands to correct this:
sudo gpg --keyserver pgpkeys.mit.edu --recv-key 8B48AD6246925553 sudo gpg -a --export 8B48AD6246925553 | sudo apt-key add - sudo gpg --keyserver pgpkeys.mit.edu --recv-key 7638D0442B90D010 sudo gpg -a --export 7638D0442B90D010 | sudo apt-key add - sudo apt-get update
Then you can finally install the certbot program. On Jessie do:
sudo apt-get update sudo apt-get install certbot -t jessie-backports
On Stretch do:
sudo apt-get update sudo apt-get install certbot
Get Yourself A Certificate
The certbot program requires that your machine serves a web page on port 80 which is accessible from the internet. This allows Let's Encrypt to check whether you are really in control of your domain name. This is not only required during the initial request for the certificate, but also when it has to be renewed (within every 90 days).
First of all make sure that the firewall on your Raspberry Pi doesn't block port 80 from the internet.
Then you might have to change the NAT port forwarding rules in your router to redirect incoming port 80 TCP traffic to port 80 of your Raspberry Pi.
In case another machine in your network is used as a web server you may have to redirect port 80 to your Raspberry Pi temporarily until the certificate has been downloaded.
In that case you can not use automatic renewal, which is explained later.
Now run one of the following commands, depending on your situation. You may have to change the webroot path as set on your server.
sudo certbot certonly --standalone --preferred-challenges http-01 -d yourdomainname
sudo certbot certonly --apache --preferred-challenges http-01 --webroot-path /home/var/www -d yourdomainname
sudo certbot certonly --nginx --preferred-challenges http-01 --webroot-path /home/var/www -d yourdomainname
When prompted you must provide your email address and accept the terms of use. Then, when everything goes as planned you should now see the following message:
Saving debug log to /var/log/letsencrypt/letsencrypt.log Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org Obtaining a new certificate Performing the following challenges: http-01 challenge for yourdomainname Waiting for verification... Cleaning up challenges Generating key (2048 bits): /etc/letsencrypt/keys/0000_key-certbot.pem Creating CSR: /etc/letsencrypt/csr/0000_csr-certbot.pem IMPORTANT NOTES: - Congratulations! Your certificate and chain have been saved at /etc/letsencrypt/live/yourdomainname/fullchain.pem. Your cert will expire on 2017-09-29. To obtain a new or tweaked version of this certificate in the future, simply run certbot again. To non-interactively renew *all* of your certificates, run "certbot renew" - If you like Certbot, please consider supporting our work by: Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate Donating to EFF: https://eff.org/donate-le
Now you can restore your previously altered firewall and router settings and restart your web server again, in case you had to disable it temporarily.
Automatically Renew Your Certificate(s)
Once your certificate has been installed it will expire in 90 days. Fortunately certbot can automatically renew certificates which are due to expire within 30 days. For that you'll have to setup a simple cronjob, which is run weekly. Enter the command sudo crontab -e and add the following lines to the end of the file:
# Renew all of our Let's Encrypt certificates before they expire 0 3 * * 6 certbot renew --noninteractive --webroot-path /home/var/www --post-hook "systemctl restart mosquitto" > /dev/null
This will run certbot every Saturday night at 3 AM. You can chose any time and day you want though.
Please note that this cronjob will update all of your Let's Encrypt certificates which will expire within the next 30 days. Each time a certificate is updated your Mosquitto daemon will be restarted. Therefore you might want to investigate a bit more how to restart the daemon only when your IOT certificate is updated.
Also note that you should be running a web server on port 80 during the renewal process. If another machine in your network is running a web server on port 80 you can not automatically renew your certificate! Fortunately the good people from Letsencrypt will send you an email about a week before expiration of your certificate so you can do the manual update process.One final note, there is no way you can test whether the automatic update is working or not until your certificate is at least 60 days old. So make sure you check the renewal process the first time to ensure that everything has been setup correctly. You'll start getting some emails from Let's Encrypt when your certificate is due for renewal, so I guess you'll notice whether auto renewal works or not.
Update: After my first renewal I noticed that I was no longer able to log in to my broker.
It failed with a certificate error, even though the certificate was renewed correctly.
I found out that I had to restart the broker to make it realize that it had to use the new certificate.
The easiest way to restart your broker is to create a small script which runs certbot and then restarts the broker.
Then setup your cron job to run this script weekly, instead of the certbot program alone.
Configure Mosquitto To Use SSL
Edit the file /etc/mosquitto/conf.d/config.conf again and add the following lines before the password lines we've already entered in the previous step:
listener 1883 localhost listener 8883 certfile /etc/letsencrypt/live/yourdomainname/cert.pem cafile /etc/letsencrypt/live/yourdomainname/chain.pem keyfile /etc/letsencrypt/live/yourdomainname/privkey.pem
The first line allows only local clients to connect to the default MQTT port without SSL encryption and without username and password, just for your convenience. The second line sets up the default MQTT secure port 8883. The other three lines are there to tell Mosquitto where to find the certificates. You'll have to change all references to yourdomainname into your own domain name of course.
Then restart Mosquitto to make the changes effective:
sudo service mosquitto restart
For me this didn't work though. I could not connect to my broker anymore, so I ran tail -f /var/log/mosquitto/mosquitto.log and saw:
1498991188: mosquitto version 1.3.4 (build date 2017-05-29 05:46:59+0000) starti
ng
1498991188: Config loaded from /etc/mosquitto/mosquitto.conf.
1498991188: Opening ipv6 listen socket on port 1883.
1498991188: Opening ipv4 listen socket on port 1883.
1498991188: Opening ipv4 listen socket on port 8883.
1498991188: Opening ipv6 listen socket on port 8883.
1498991188: Error: Unable to load CA certificates. Check cafile "/etc/letsencrypt/live/yourdomainname/chain.pem".
The easiest way to solve this problem is to make two directories in /etc/letsencrypt world readable.
This might be bad practice on a machine with multiple users, but I don't care about that because I'm the only user on this machine.
And the only bad thing that can happen from this is that an intruder steals your certificates and sets up a new server to pretend to be your broker machine.
Run these two commands to fix it:
sudo chmod 755 /etc/letsencrypt/live sudo chmod 755 /etc/letsencrypt/archive
Finally you'll have to setup your firewall to accept incoming TCP traffic to port 8883. And you'll have to forward port 8883 to your broker machine in your NAT router forwarding rules.
This basically completes the secure setup of Mosquitto. We can test this now, but chances are that the test will fail. I'll explain why they may fail and what you can do about it below.
Let's try it out and see if it works or fails. In terminal 1 execute the command:
mosquitto_sub -h yourdomainname -t test -u username -P "secret" -p 8883 --capath /etc/ssl/certs/
And in the second terminal type:
mosquitto_pub -h yourdomainname -t test -m "hello world" -u username -P "secret" -p 8883 --capath /etc/ssl/certs/
If everything works you should see the message "hello world" appear in the first terminal screen.
Please note that you'll have to use your domain name now to connect to your broker because the certificate is only valid for that domain name. The --capath part instructs Mosquitto to use SSL encryption and tells the program where to find the root certificates on your machine (this path may differ on different OSses). Without it Mosquitto will attempt to connect without SSL, regardless what port number is used.
Networking Issues
If the last example failed things might now get quite complicated.
Or even if you don't use SSL encryption you may find that your mobile phone can only connect to your broker when you're at home, not while you're roaming around.
Why?
Because traditional consumer routers are stupid.
The problem is that a DNS request to your domain name will produce your own public IP address.
This is fine when you are not connected to your own network, when you're not at home.
However, when you're at home your router doesn't know how to access its own public IP address.
It is too stupid to replace that public IP address with the private IP address of your broker machine.
Some routers are smart enough and use so called hairpin NAT, also known as NAT loopback, to do the proper translation for you.
My Fritz!box for instance does that and I don't have any problems with it.
Another problem you may encounter in your home network is when your ISP uses dynamic IP addresses, where your public IP address can change regularly.
There are a couple of possible solutions for you. It's up to you to decide which solution works best for you.
- Buy yourself a better router. Obviously you should find one which can do hairpin NAT or NAT loopback. Usually these routers are from the more expensive segment of the router market. The dumb thing is that router manufacturers usually don't tell you that they have this feature, making it difficult for you to pick the right one.
- Try installing DD-WRTon your router. This will likely void your warranty though.
- Run a local DNS server. Then you can setup this DNS server to translate your domain name to a local IP address for your local network only.
- Use the /etc/hosts file of your OS to translate your domain name to your broker's local IP address. This only works on your computers, not on your phone for instance. And it doesn't solve your roaming problem on your phone either.
- Disconnect your phone from the Wifi at home. This is only feasable if you have a large enough mobile data plan of course. Otherwise watching Youtube on your phone may get a little expensive.
- Use a dual setup on your phone. Connect to the local setup when you're at home and to the remote setup when you're away. It could work, although it is not very convenient I guess.
- If you're unfortunate enough that you have a dynamic public IP address, then you might get a domain name from one of the Dynamic DNS providers. They use some form of a client on a machine in your private network to automatically change the DNS entry for you.
- Run your Mosquitto broker on a VPS machine somewhere on the internet. This is by far the best solution. It solves both the roaming problem and the dynamic IP address problem for you. However this solution comes with some costs. The first cost is that you are now forced to care for this VPS, keep it updated and safe. The second cost is that you'll have to pay for that VPS, although they can be very cheap.
Recommended VPS Hosters
As a side track I can highly recommend the following two Virtual Private Server hosting providers.
A VPS can solve all of the above networking issues for you.
And as a bonus you can run all your web sites on it, use it as VPN server and run lots of other tasks for you.
I use both of these VPS hosters for quite some time now, with great satisfaction.
In fact this page is served by 2 machines, one from each VPS hoster.
This time the page was served by
Digital Ocean's platform is very stable and mature.
There are a lot of easy to install operating systems to choose from.
You can even deploy preinstalled applications with just a few clicks.
And what does it cost to run a VPS server?
The cheapest one, which I use, costs only $5 per month, ex VAT.
Even at that price the server outperformed the web server of my previous web hoster 3 times!
Payment is done via a prepaid credit, per hour you run the server(s).
You can simply spin up a server within minutes, play with it for a while, destroy it a few hours later and it's going to cost you some $0.10 or so.
Sign up using this link, pay $5 via credit card or Paypal and you'll receive $10 referral credit. So for just $5 you can play with a VPS for about 3 months. If you don't like it (which I doubt), simply don't pay more. No strings attached.
Scaleway is even cheaper than Digital Ocean, delivering more processor power, more RAM and more SSD storage capacity for as low as €3.00 ex VAT per month!
For that price you can even get a dedicated bare metal ARM machine, all for yourself.
An extra advantage is that your internet bandwidth is unmetered.
You can download/upload as much as you like, at 200Mbit/s.
The disadvantage for you may be that currently there are only 2 locations to choose from, Paris and Amsterdam.
At first, when I started using Scaleway, there were some small teething issues, which have all been resolved by their capable and fast support team. I can safely say that their platform now is as mature and as stable as that of Digital Ocean.
Payment is due at the end of the month, via credit card.
Here you also pay per hour that your servers have been running.
So you can also spin up a server, play with it for a couple of hours, destroy it and pay some $0.10 at the end of the month.
Don't forget to destroy your IPv4 address of your machine too, otherwise you keep paying for it (€1 per month).
And again, if you don't like it (which I doubt), destroy all of your machines and IPv4 addresses and you'll be charged one last time at the end of the month, and that's it.
Again no strings attached.
Final config.conf File
I've promised you my final /etc/mosquitto/conf.d/config.conf file so here it is:
# See mosquitto.conf(5) for more information. max_queued_messages 200 message_size_limit 0 allow_zero_length_clientid true allow_duplicate_messages false autosave_interval 900 persistence true persistence_file mosquitto.db listener 1883 localhost listener 8883 certfile /etc/letsencrypt/live/yourdomainname/cert.pem cafile /etc/letsencrypt/live/yourdomainname/chain.pem keyfile /etc/letsencrypt/live/yourdomainname/privkey.pem allow_anonymous false password_file /etc/mosquitto/passwd
Don't forget to restart your Mosquitto server after you've made any changes to this config file. This file not only includes what we've discussed so far. It also includes some sensible settings, according to the information I've found on the internet. If you want to do more tweaking yourself you can always consult the man mosquitto.conf man page.
PS: I have noticed that Mosquitto fails to start when you use settings it doesn't understand, often without explaining to you why in the log file. Therefore it is wise to test your server after each update to the config file.
Bonus Encore
Do you see the listener 1883 localhost line in our final config file above?
This line opens up very interesting possibilities when you run your Mosquitto broker on a VPS machine, somewhere on the internet.
It allows you to connect to your broker without using SSL and username and password, only when you run the client on the broker machine itself.
Why is that so interesting?
I hardly ever run clients on my broker machine.
Well there is a way around this.
You probably have a Raspberry Pi in your local network at home, which runs 24/7 anyway.
Let's use this to tunnel into our broker machine.
That way you can simply connect to your broker from your home network, without a username and password and without the certificate nonsense.
This not only simplifies the connection of clients which are bound to your local network anyway, it may even be more secure.
You don't have to store passwords in plain sight on your clients anymore.
And small clients, which can't use encrypted connections, can also be connected to your highly secure broker again, thanks to this ssh tunnel.
So what do we need?
We need ssh access on the remote, secure, broker machine, which we already have because it is our own VPS machine.
We also need a Linux computer in our home network, preferably one which is running 24/7.
A Raspberry Pi will do.
And we need a way to setup a persistent tunnel from this Raspberry Pi to our broker machine.
Ah, we don't have that last part yet.
So let's get to it:
sudo apt-get install autossh
For convenience sake we're going to use the ~/.ssh/config file. Use your preferred editor to edit this file and add the following lines at the end of it:
Host iot
HostName yourbrokermachine
User username
LocalForward 0.0.0.0:1883 localhost:1883
ServerAliveInterval 30
ServerAliveCountMax 3
After that you can start the tunnel with:
autossh -M 0 -f -T -N iot
From then on you can close the terminal and forget about the tunnel. Autossh will see to it that the tunnel remains open. If the connection to the broker gets broken, for whatever reason, autossh will reconnect it, for as long as the Raspberry Pi keeps running and the internet connection is alive.
Now all the clients in your home network can connect to your broker, through the tunnel, without SSL.
mosquitto_sub -h yourraspberrypi -t yourtopic
