MQTT Conversion, First Steps

Last post I talked about trying out mqtt and being impressed with the ease of using it. It really went well and I decided to convert my house monitoring system to using this tool. So I took the very first steps. This is going to be a complicated conversion because I already have a system running and I don't want to break it while I'm moving to something different, so I decided on a few things to get me started.

I'm going to divide the sensors, saving data, and presenting data into different things. For example, I'm starting with the weather station because it's the simplest, but still has multiple sensing devices. The entire house is published under the 'topic' Desert-Home, so the weather station will be Desert-home/Weather, and the two current sensors are the Acurite 5n1 and the barometer out on the fence. The interesting thing here is that they are radically different.

The 5n1 is detailed on about a hundred posts on this blog, but finally wound up being a sensor on the roof that transmits in the 933MHz range. I catch and decode the signal with an SDR (software defined radio) and some code I stole and heavily modified. The readings from this sensor will be published under Desert-Home/Weather/5n1.

The barometer is a device I built up myself and put in a Stevenson Screen out on a fence post in the yard. It has an XBee that sends a JSON string to a Pi. This will be published under Desert-Home/Weather/Barometer.

I can see each of these devices by simply subscribing to 'Desert-Home/Weather/#'. The pound sign means to get everything that comes in under 'Desert-Home/Weather' and that will include every sensor I come up with related to weather.

I also decided to use JSON strings as much as possible for a few reasons. Most important to me is the ease of reading a JSON string when you're monitoring the network to see why something is messing up. The JSON string has key:value pairs that tell you what it is and what the value is. This helps a lot when I'm looking for a problem. This will be painful since most of my sensors use another technique entirely and they will have to be converted. Another reason important to me is that I don't have to write parsers to get the data back into variables, There are libraries that will do that for me and all I have to do is use them. Maybe that will make up for the pain I go through converting the devices.

So, since the XBee network I have is on a different Pi than the weather station RF software, I get to try out a multi-machine solution as the very first thing I try. Step one is done; I brought up a mqtt server (mosquitto, see previous post) and added publish lines to the XBee receive code and the weather station decoding. It worked on the second try; I had a couple of syntax errors to fix.

Mosquitto caught the data and all I have to do to see it is use the tool mosquitto_sub to watch the data as it is generated. The command line:

 mosquitto_sub -h "192.168.0.205" -t "Desert-Home/Weather/#" -v

shows me what is coming in as it happens:

Desert-Home/Weather/Barometer {"Barometer":{"temperature":"114.4","pressure":"1011.4","utime":"1434639446"}}
Desert-Home/Weather/5n1 {"sensorId":{"SID":"92","t":"1434664650"},"channel":{"CH":"A","t":"1434664650"},"messageCaught":{"MC":"0","t":"1434664650"},"battLevel":{"BAT":"7","t":"1434664650"},"windSpeed":{"WS":"6.0","t":"1434664650"},"windDirection":{"WD":"W","t":"1434664650"},"temperature":{"T":"114.6","t":"1434664632"},"humidity":{"H":"9","t":"1434664632"},"rainCounter":{"RC":"445","t":"1434664650"}}
Desert-Home/Weather/5n1 {"sensorId":{"SID":"92","t":"1434664650"},"channel":{"CH":"A","t":"1434664650"},"messageCaught":{"MC":"0","t":"1434664650"},"battLevel":{"BAT":"7","t":"1434664650"},"windSpeed":{"WS":"6.0","t":"1434664650"},"windDirection":{"WD":"W","t":"1434664650"},"temperature":{"T":"114.6","t":"1434664632"},"humidity":{"H":"9","t":"1434664632"},"rainCounter":{"RC":"445","t":"1434664650"}}
Desert-Home/Weather/5n1 {"sensorId":{"SID":"92","t":"1434664650"},"channel":{"CH":"A","t":"1434664650"},"messageCaught":{"MC":"0","t":"1434664650"},"battLevel":{"BAT":"7","t":"1434664650"},"windSpeed":{"WS":"6.0","t":"1434664650"},"windDirection":{"WD":"W","t":"1434664650"},"temperature":{"T":"114.6","t":"1434664632"},"humidity":{"H":"9","t":"1434664632"},"rainCounter":{"RC":"445","t":"1434664650"}}

Yes, the outside temperature is REALLY 114.4, remember I live in Arizona and this is the middle of June, but it's a dry heat, the humidity reading is 9%. As if that matters when the temperature is this high ...

But, also notice that I have both strings coming to the subscription even though they are very different devices, on different Pi's. Heck, they could be half way around the world and it wouldn't matter, it would still work. I tested that with a conspirator just this morning.

Now to follow my plans, I have to catch this data in some other process and save it to my database. I'll get to that soon, maybe even today, but I'm going to hold off on changing the presentation to the web for later because I have many different presentations to consider when I get to that point. See, currently I take everything presented from my data bases. If I'm using my Android app to look at the house, the data is coming out of the data base. When I send a command to close the garage doors, the command is executed and the app doesn't update until it is recorded in the data base. I may change that to reflect the latest reading from the mqtt server. We'll see, and then I'll probably change it.

The code is not in GitHub, I'm going to hold off until I get enough in there to make a significant difference. But, the example from the last post is exactly what I did with the difference above. This is a lot simpler and more versatile than I expected.