The previous entry on this project is here <link>.
I worked on this project earlier this year and put it into daily service back in April. The batteries died last night (this morning) at 4:00 AM, earlier than expected. They lasted 5.5 months and had the same rapid decline to 2.7 volts that I saw when I was running it in accelerated mode to understand how to do this.
Basically they coasted along dropping a little over time and then just dived, relatively speaking over a couple of weeks to a level where it wouldn't work anymore. This seems to be the normal behavior of alkaline batteries, so nothing new.
I am disappointed that it took less than six months to die though, but it died two days after my TV remote control died and I changed those batteries around the same time as this device. I remember because of the coincidence of having the remote need batteries around the same time as I started the long term test on the sensor. Could be the life of the cheap batteries I'm using.
The device was set for 115 seconds off and 5 seconds on and I can easily change that to a much longer off period if I need to, but I'm not sure I want to. Having the temperature available at two minute intervals is nice and fits with my longer term plans for temperature control in the house.
On the reliability front, this thing has worked day after day without a hiccup. I use it to turn off the bedroom and outside lights when I go to bed every day and sometimes just to show it off. Never gave me a problem unless the controller it talks to was having problems because I changed something there. It's still a bunch of components on a proto board though. I couldn't find an enclosure to fit it that I liked and decided to wait until I could buy a 3D printer to make something for it.
My tractor and the terraforming work around the house ate up the money I was setting aside for the printer and that has been postponed until I finish that stuff. Keeping flood water out of my house and repairing ruts in the driveway will always take precedence.
I put some more of the exact same batteries in it, and it's beside the bed again doing exactly what it's supposed to. I'll check on it from time to time and see what develops over the next six months or so.
Maybe by then I'll have an enclosure for it.
I live in the Arizona Desert, Southwestern USA. It gets hot here, and my power bills got out of hand. This is a journal of my various efforts to bring this problem under control using the cheapest technology I could find. Saving money shouldn't cost a fortune.
Monday, November 2, 2015
Monday, September 7, 2015
Yes, I'm alive
I've been suffering from blog withdrawal, and folk have contacted to make sure I'm still alive. So, I thought I'd at least write a bit about what I've been doing recently.
I've been doing things that aren't of a technical nature. I've dug some trenches and holes with the new (to me) tractor, and put a lot of money into my other tractor to get it running again. Here's a picture of my agricultural fleet:
I got the little tractor when I first moved out here and have been fixing things on it over time. It died recently because the fuel pump wore out. If you're familiar with diesel engines, you just cringed a bit. Diesel engine fuel pumps are expensive, and some of them are a real pain to replace because they set flow with shims and timing with various techniques. Fortunately, this is a three piston pump with each piston supplying an injector so there's no timing needed. I did have to set the flow with shims, but I guessed right the first time.
The pump cost me $535 and then I had to replace other stuff as well since messing with the old hoses and things broke stuff. All in all I have another $650 or so invested in it, but I need the little tractor to fit in places that the yellow monster can't get to. When I was all done, it started when the first cylinder hit the compression cycle; perfect.
Now the little tractor is off the to-do list, the trench is good enough to keep the flood waters from getting to the house, so I'm moving the big tractor into the garage for some work. Someone before me decided to fix the dashboard on it and got in over their head. None of the gauges work, and I want to know the oil pressure and temperature at an absolute minimum. I couldn't care less about the total hours, but it would be nice to know the battery is charging. I guess I get to learn all about monitoring tractor vital statistics next.
And yes, i can fit the tractor into the garage. I have an empty RV garage that I built with the house, but then couldn't actually afford to keep the RV on the road because fuel prices went totally out of sight. I sold the RV and use the space to set up tools for various projects. It's really nice to be able to set a table saw beside a band saw and move from one to the other as I build something.
A man can't ever have a big enough garage.
I really haven't done squat on the technical aspects of the house. Yes, I keep experimenting with MQTT to understand more about it and I'm very slowly converting the various devices to use the toolset, but progress is slow because I keep thinking about hydraulic pressures and depth of culvert openings to handle water flow.
I'm easily distracted.
Speaking of culverts: The driveway to my place kept getting a deep rut dug into it by the monsoons here. See, we don't get much rain, but when it does rain, sometimes it comes down in a real torrent. We sometimes get three inches in twenty or so minutes and that causes flash flooding. I'm well away from the most dangerous areas for this, but as last year would attest, I'm not invulnerable to water coming down a hill and making a mess:
You should be able to see how high the water got on the house in the picture. When I realized I couldn't keep the water out of the house, I just opened the doors on the other side and let it out. That left me with about a half inch of water in the house and my wet vacuum was able to clean up the mess in a couple of days. I have tile floors, so all I had to do was mop up and everything was fine. It took about three weeks to clean out the pool though. I got seven wheel barrows of mud out of it before I was done.
Anyway, back to the culvert. When I did some digging around the rut in the road, I found caliche near the surface. That limited the size of culvert pipe I could put in, so I used three 10" pipes instead of a single 18" pipe. Metal culverts need some sand in the bottom and then a covering with something that didn't have rocks and could be compacted. The sand I had in abundance, but I had to get 30 ton of aggregate to fill it back in. It turned out nice and the 16" rut is gone from the drive. The driveway services four houses, so I did a good deed for the neighbors as well.
I want to admit to something though; I didn't use my yellow tractor to dig the culvert channel. The big yellow tractor was brand new to me and I didn't have a clue how to use it. I begged a neighbor for a favor and got him to bring his big ol' Cat backhoe over and help out.
Then another neighbor from down the road came up to watch and brought his skip loader with him. I had an entire crew of folk out there messing around with the road. I totally love my neighbors! It took all day because we dug up a telephone line (mine) and had to wait for them to show up to advise us. Yes, they were responsible for it because I called in advance and had the lines marked and they missed it. It really does pay to follow the rules when your digging with a machine; if I hadn't had it marked, they would have made me pay for fixing the line.
The before picture:
It's hard to tell from the picture, but the right hand side of the rut is over 16 inches deep. It made for an interesting drive for people with normal cars; they had to pull way over to the side and go very slow. The paint marks are where they marked the various lines for power and phone. They missed.
That gray thing in the hole is what's left of the conduit and MY phone line. It was a good eight feet from where they marked it should have been. The picture below is just before I was finally able to cover the pipe. It took them over a week to decide who was going to come out and fix it. I finally got them to do something when I told them it was just a matter of time before someone drove into the hole and decided to sue because of the damage to their vehicle. I think the idea of lawyers was enough for them to actually do something.
Just before I was finally able to cover it:
The folk among you that have done something like this are wondering what I did with the dirt and rock I dug out of the hole since I refilled it from other sources. I used that dirt to fill in the channels on my lot that were carved by the rain. I still need some more, but that will have to wait until I have some more money. With the new trench to divert the water, I don't mind buying some fill material to spruce thing up a bit and cover some of the rocks. Yes, I used my own tractor for this with a little help from the skip loader neighbor; I can work a front loader just fine, it was the backhoe that I needed to learn about.
Looks like a normal road now:
The rocks on the right are to keep people from driving into the intake hole, there will be more rocks moved in to define the road better when I get around to it. I didn't want someone trying to cut the corner driving into the hole there. Now, they'll slam into the rock and have only themselves to blame.
Folk in apartments in town wonder why people like me live out here and put up with this kind of thing. Frankly, from time to time I wonder myself. Me and Barkley, my dog, have killed three rattlesnakes so far this year; the desert toads have been going through their breeding season and the screams they make actually hurt your ears; there have been several weather alerts for sand storms; only one scorpion sting so far; and I had to clean the septic filter yesterday. But, sometimes you get something like this:
or
or
and think, "Maybe this isn't so bad after all."
Remember back in school when the teacher wanted an essay about "What I did last vacation?" Well this is mine.
I've been doing things that aren't of a technical nature. I've dug some trenches and holes with the new (to me) tractor, and put a lot of money into my other tractor to get it running again. Here's a picture of my agricultural fleet:
I got the little tractor when I first moved out here and have been fixing things on it over time. It died recently because the fuel pump wore out. If you're familiar with diesel engines, you just cringed a bit. Diesel engine fuel pumps are expensive, and some of them are a real pain to replace because they set flow with shims and timing with various techniques. Fortunately, this is a three piston pump with each piston supplying an injector so there's no timing needed. I did have to set the flow with shims, but I guessed right the first time.
The pump cost me $535 and then I had to replace other stuff as well since messing with the old hoses and things broke stuff. All in all I have another $650 or so invested in it, but I need the little tractor to fit in places that the yellow monster can't get to. When I was all done, it started when the first cylinder hit the compression cycle; perfect.
Now the little tractor is off the to-do list, the trench is good enough to keep the flood waters from getting to the house, so I'm moving the big tractor into the garage for some work. Someone before me decided to fix the dashboard on it and got in over their head. None of the gauges work, and I want to know the oil pressure and temperature at an absolute minimum. I couldn't care less about the total hours, but it would be nice to know the battery is charging. I guess I get to learn all about monitoring tractor vital statistics next.
And yes, i can fit the tractor into the garage. I have an empty RV garage that I built with the house, but then couldn't actually afford to keep the RV on the road because fuel prices went totally out of sight. I sold the RV and use the space to set up tools for various projects. It's really nice to be able to set a table saw beside a band saw and move from one to the other as I build something.
A man can't ever have a big enough garage.
I really haven't done squat on the technical aspects of the house. Yes, I keep experimenting with MQTT to understand more about it and I'm very slowly converting the various devices to use the toolset, but progress is slow because I keep thinking about hydraulic pressures and depth of culvert openings to handle water flow.
I'm easily distracted.
Speaking of culverts: The driveway to my place kept getting a deep rut dug into it by the monsoons here. See, we don't get much rain, but when it does rain, sometimes it comes down in a real torrent. We sometimes get three inches in twenty or so minutes and that causes flash flooding. I'm well away from the most dangerous areas for this, but as last year would attest, I'm not invulnerable to water coming down a hill and making a mess:
You should be able to see how high the water got on the house in the picture. When I realized I couldn't keep the water out of the house, I just opened the doors on the other side and let it out. That left me with about a half inch of water in the house and my wet vacuum was able to clean up the mess in a couple of days. I have tile floors, so all I had to do was mop up and everything was fine. It took about three weeks to clean out the pool though. I got seven wheel barrows of mud out of it before I was done.
Anyway, back to the culvert. When I did some digging around the rut in the road, I found caliche near the surface. That limited the size of culvert pipe I could put in, so I used three 10" pipes instead of a single 18" pipe. Metal culverts need some sand in the bottom and then a covering with something that didn't have rocks and could be compacted. The sand I had in abundance, but I had to get 30 ton of aggregate to fill it back in. It turned out nice and the 16" rut is gone from the drive. The driveway services four houses, so I did a good deed for the neighbors as well.
I want to admit to something though; I didn't use my yellow tractor to dig the culvert channel. The big yellow tractor was brand new to me and I didn't have a clue how to use it. I begged a neighbor for a favor and got him to bring his big ol' Cat backhoe over and help out.
Then another neighbor from down the road came up to watch and brought his skip loader with him. I had an entire crew of folk out there messing around with the road. I totally love my neighbors! It took all day because we dug up a telephone line (mine) and had to wait for them to show up to advise us. Yes, they were responsible for it because I called in advance and had the lines marked and they missed it. It really does pay to follow the rules when your digging with a machine; if I hadn't had it marked, they would have made me pay for fixing the line.
The before picture:
It's hard to tell from the picture, but the right hand side of the rut is over 16 inches deep. It made for an interesting drive for people with normal cars; they had to pull way over to the side and go very slow. The paint marks are where they marked the various lines for power and phone. They missed.
That gray thing in the hole is what's left of the conduit and MY phone line. It was a good eight feet from where they marked it should have been. The picture below is just before I was finally able to cover the pipe. It took them over a week to decide who was going to come out and fix it. I finally got them to do something when I told them it was just a matter of time before someone drove into the hole and decided to sue because of the damage to their vehicle. I think the idea of lawyers was enough for them to actually do something.
Just before I was finally able to cover it:
The folk among you that have done something like this are wondering what I did with the dirt and rock I dug out of the hole since I refilled it from other sources. I used that dirt to fill in the channels on my lot that were carved by the rain. I still need some more, but that will have to wait until I have some more money. With the new trench to divert the water, I don't mind buying some fill material to spruce thing up a bit and cover some of the rocks. Yes, I used my own tractor for this with a little help from the skip loader neighbor; I can work a front loader just fine, it was the backhoe that I needed to learn about.
Looks like a normal road now:
The rocks on the right are to keep people from driving into the intake hole, there will be more rocks moved in to define the road better when I get around to it. I didn't want someone trying to cut the corner driving into the hole there. Now, they'll slam into the rock and have only themselves to blame.
Folk in apartments in town wonder why people like me live out here and put up with this kind of thing. Frankly, from time to time I wonder myself. Me and Barkley, my dog, have killed three rattlesnakes so far this year; the desert toads have been going through their breeding season and the screams they make actually hurt your ears; there have been several weather alerts for sand storms; only one scorpion sting so far; and I had to clean the septic filter yesterday. But, sometimes you get something like this:
or
or
and think, "Maybe this isn't so bad after all."
Remember back in school when the teacher wanted an essay about "What I did last vacation?" Well this is mine.
Thursday, July 9, 2015
Unexpected Electronics
I have a 2002 Jeep Wrangler that is my everyday car. Living where I live I occasionally actually need the 4x4 and the winch I put on it has helped several people out of mud holes; nice vehicle. The other day my dash check engine light came on and when I glanced at the gauges, the oil pressure was maxed out.
I've seen this before on other vehicles, and wasn't too worried; if it had gone the other way, I would have pulled off the road and checked it out. So, when I got to the WalMart down the road to pick up some stuff, I killed the engine and then turned the key on without starting it. Yep, I had 80 pounds of oil pressure without starting the engine. Needed a new sensor.
Stopped off at O'Reilly's and got one, then went home. I decided to put it in later when the engine was cold, I had other things I wanted to do besides burn myself on the exhaust pipe replacing a sensor.
A couple of days later I went out with my morning coffee, a dog, and intentions of knocking this repair off in about thirty minutes. An hour later, I still hadn't managed to get the dog gone plug off the sensor. Yes, I read the articles on the web, and looked a videos and such, but they didn't help a bit; the stupid plug was still on there. The sensor was near the passenger side firewall about a yard away and I could barely reach it cramming my arm down among all the wires and such with the release for the plug on the wrong side where I couldn't see it and get a screw driver in for leverage. Take a look:
Can't see it? Neither could I at first. It's that brass looking sliver thing in the middle of the picture way down inside. Click on the picture then enlarge and you'll see it. No, I couldn't get to it from the bottom; there was exhaust pipes and suspension in the way and I couldn't get my arm in there.
I was pretty disgusted by this time so I went Medieval on it. I grabbed a steel bar, a hammer and busted the plastic top off the sensor. Now, I pulled the wire up and disconnect the plug from the broken piece. I was on my way and had the job done in about fifteen minutes.
Moral, when all else fails, get the hammer.
This is all introduction to what I really wanted to show you; all folk like us have had problems like this where something is at the end of our reach and everything conspires to keep us from doing it. Just get a hammer.
What I wanted to show you was the circuitry I found:
Yes, inside the sensor sealed in brass and plastic is a circuit board. I couldn't find any numbers inside to tell me about the circuitry, but it was reasonably sophisticated. No wonder the thing cost me $48; this is a cool device.
It made me wonder about some of the other sensors the car has. The throttle position sensor is a simple potentiometer; I've already taken one of them apart. I'm going to dismantle the others as they fail and get replaced, it may make this kind of job more fun.
Yes, I know that I have other things to do. I'm getting to the conversion to mqtt as I have the time, but converting the various devices to use JSON strings and such is a drudge since I've done that before, but I'll get to it ... promise. But I've got that new tractor and some holes to dig before the end of August when the real rains come.
Have fun.
Saturday, June 27, 2015
Arduino, Raspberry Pi, Pool Controller and Another Guest Speaker
Let me introduce Mike He's another one of us folk that wanted automation, but didn't want to pay unreasonable prices or be locked into the control of some system 'out there.' He's been working on a project that I want to get into deeply at some point (when I get sorta caught up), and he's way ahead of me on it. Go ahead Mike:
The initial project...
Several years ago, I was trying to think of a way to extend my swimming season without breaking the bank. I have a heat pump on the pool but they aren't very cost effective to operate. As I live in Florida, we have plenty of sun so solar was an obvious choice. Problem is that I didn't want my roof covered with plastic pool panels as I intend to install PV electric panels there and since they tend to develop leaks, i didn't want salt water on the roof. I already heat my hot water with solar so I started looking into using the same concept for the pool.
I located a pair of glazed DHW panels and set them up with a titanium heat exchanger so that I wasn't running corrosive salt water through the panels. Here's a picture of the panel...
I used a circulation pump to pump the water in the closed solar loop and quickly discovered I could gain 5 to 6 degrees a day in pool temperature in March. I was losing 2 to 3 degrees at night but the idea was successful as I was still gaining heat. Since I didn't want to heat the pool past a certain point and I didn't want to have to manually control it, I needed a contoller. I was already toying with my first Arduino (a Uno) so it was an obvious choice.
I began to research temperature sensors and ran across the "Differduino" project. All I had to do was modify the hell out of it and I had my basic controller! I added an ethernet shield so that I could send data to an external site now known as Xively.
The controller monitors the pool temperature and the panel temperature and turns the circulation pump on and off on a differential provided the pool temperature is below the setpoint. This part of the project has not changed, much....
The evolution begins...
Then one day, I stumbled across the "Desert Home" page. Don't remember what I was looking for but this crazy fool was doing things I had thought about but hadn't figured out how to do practically. As he had put up all of his code and wanted people to see it and benefit from it, I decided to do so. Dave was (and still is) very helpful in getting things up and running and I soon had a Raspberry Pi to play with.
The Ethernet interface on the pool controller was replaced with an XBEE radio and the data was being magically sent through the air to the RPI where it was being stored to do with as I chose. This led to a Web interface that can be seen at myeager.no-ip.biz that allowed me to see the data in real time. I added a temperature and humidity sensor so I knew what was going on outside. Here's a picture of the controller about this time..
The house controller...
The heart of the project has actually shifted from the pool controller to the house controller. This creation started out as Dave's, 100%! As it was written in Python, it was a learning curve. My programming experience was mostly C++. As I came to figure out more of Dave's code, it became more natural to read and much easier to learn from. Once again, Dave would prod me along rather than hand me the answers. Kudos to Dave for that...
As I was collecting more data from the pool controller and have built a few other devices as well, data was getting tedious to process. I ran across a library called ArduinoJSON and spent a weekend in the woods figuring out how it best suited my needs. I'd like to think I turned Dave on to that little treasure but he may have been exploring JSON already. If you haven't looked at JSON, I suggest doing so. I learned quickly that only so much will fit on an Uno...
Reboot!!!
I got hold of a Mega 2560 and loaded the code to it. I had a ton of memory now (and 3 extra hardware serial ports). The SoftwareSerial library I was using could now go away but I didn't do that until I realized that it didn't work well with two way communication on the Mega. I added and GPS module I scavenged from Streets and Trips. Now I had Dave's house clock as well so I could set up timers for the pool. A Timezone library even correctly handled the time change twice a year. This has come in handy on a few other devices. I intend to add a pH probe to monitor the pool and an acid pump to keep the pH at a certain point but I haven't done it yet. I have added two way communication with the pool controller and several things are now adjustable from the Web interface.
The pool controller and all other devices I've built so far operate entirely on their own and continue to do their thing even if the house controller is offline. I recently had to replace my pool pump motor and chose to get a Hayward variable speed unit instead. Best decision possible in my opinion. Much quieter and the energy efficiency is a big benefit. Electric bill went down almost $60! I'm now using the timer that I programmed to run the pool pump to run the chlorine generator and the timers on the pump to vary the pump speed. I let it run at a low speed overnight instead of turning it off.
The house controller also now incorporates ZWAVE and controls several lights and doors. It's function grows almost every time I look at it. The power of these inexpensive machines is impressive.
What's next?
The pH monitor and acid pump are still on the list. I need to get my panels finished up, right now I'm operating on just one, and I need to get them permanently mounted. I'm working on replacing my DHW controller with one that integrates into this system and I've got an LED sign that displays weather data and pool information all over the XBEE network. I'm also looking into adding an FPH system from Hotspot. I'll do my own controller rather than using theirs but between it and the solar, I should be able to swim almost year round. We don't get much of a winter....
This is Dave again. Nice job Mike, and thanks for chiming in. See people, you're not the only one that does this kind of thing.
Have fun.
Friday, June 26, 2015
My Newest Old Toy
I've gotten some flack, not a lot, but there have been a few people that seem to think all I do is huddle over a laptop and play with tiny devices ignoring the world around me. Most of that is true, but there is another side I don't post much about.
After the flooding last year, and the possibility of the same when the rains come this year, I decided to dig some drainage ditches. My little Yannar landscape tractor can't do that in this soil. The dirt here has settled into a rock strewn landscape left over by glaciers and is almost impossible to dig in with a pick and shovel. I posted about this after a really annoying experience <link>, and I've had several more problems like that; I live on rock with a little sand thrown in.
Time for a bigger tractor because renting one at the pace I work (slowly) would be horribly expensive, and hiring a contractor has all the usual problems of getting someone to actually show up out here and do the job. There's about a hundred blog posts possible on that one subject alone.
Let me show off my cool new old toy:
This is the industrial model of the Ford 3550 tractor. These tractors were designed for daily use on large farms back in the '60s and '70s, so it's not a spring chicken, and has seen some use. However, it's supported by a huge collection of parts and books as well as a large number of people like me that have one to tinker with. This guy was made in April of 1974 by the afternoon shift; I decoded the date sticker under the hood.
This is one of the old-school tractors that doesn't use sheet metal, it uses 3/16 and 1/8 inch plate steel. Opening the hood is an interesting experience after playing with a small Japanese landscape tractor, the engine is massive.
Frankly, it scares me. Sitting 5 feet off the ground on a big machine is a thrill and the noise is exciting (remember, I ride a Harley too), but digging and moving hundreds of pounds of rock at a time is intimidating to the inexperienced. I guess I'll start by digging several holes and filling them back up to get some experience with it.
No, I'm not changing the direction of this blog, it's still about automating the house in a practical fashion with inexpensive technology that is fun to mess with, but I'm like a kid at Christmas with this big old thing and had to show it off. Besides, I may need to run wires underground or move a big device of some kind.
I think I'm going to love it.
After the flooding last year, and the possibility of the same when the rains come this year, I decided to dig some drainage ditches. My little Yannar landscape tractor can't do that in this soil. The dirt here has settled into a rock strewn landscape left over by glaciers and is almost impossible to dig in with a pick and shovel. I posted about this after a really annoying experience <link>, and I've had several more problems like that; I live on rock with a little sand thrown in.
Time for a bigger tractor because renting one at the pace I work (slowly) would be horribly expensive, and hiring a contractor has all the usual problems of getting someone to actually show up out here and do the job. There's about a hundred blog posts possible on that one subject alone.
Let me show off my cool new old toy:
This is the industrial model of the Ford 3550 tractor. These tractors were designed for daily use on large farms back in the '60s and '70s, so it's not a spring chicken, and has seen some use. However, it's supported by a huge collection of parts and books as well as a large number of people like me that have one to tinker with. This guy was made in April of 1974 by the afternoon shift; I decoded the date sticker under the hood.
This is one of the old-school tractors that doesn't use sheet metal, it uses 3/16 and 1/8 inch plate steel. Opening the hood is an interesting experience after playing with a small Japanese landscape tractor, the engine is massive.
Frankly, it scares me. Sitting 5 feet off the ground on a big machine is a thrill and the noise is exciting (remember, I ride a Harley too), but digging and moving hundreds of pounds of rock at a time is intimidating to the inexperienced. I guess I'll start by digging several holes and filling them back up to get some experience with it.
No, I'm not changing the direction of this blog, it's still about automating the house in a practical fashion with inexpensive technology that is fun to mess with, but I'm like a kid at Christmas with this big old thing and had to show it off. Besides, I may need to run wires underground or move a big device of some kind.
I think I'm going to love it.
Tuesday, June 23, 2015
Hacking Into The Iris Door Sensor, Part 4, Resolution
The previous post on this project is here <link>.
Well, my partner in questionable activity in hacking the Iris Contact Switch and Key Fob has gotten his devices to work as well as mine, so it's time to close this project off for a while.
The reason my setup worked well and his didn't was because he didn't have a router on his network of Iris devices and I did. The Iris Smart Switch is a router and I have a handful of them scattered around the house. When he plugged one in and tried the devices, his started working and away he went. I just got word that he is switching his network over to completely local control.
Why is a router necessary? Frankly, I'm not sure at this point and I'll check into it more over time, but I have a couple of door switches working quite well, and the key fob controlling one of my smart switches. The system works nicely.
Between the two of us we managed to decode the various timers and such so you folk can pick up where we left off and implement these little devices in your own home for whatever you want. Nicely made (physically) product that was hampered only by the special code Iris put in them to force you to use their hub. I don't know why manufacturers insist on doing that, especially since there are folk like me that will take it as a challenge.
Here's the latest code with the various items. It will support the Iris smart switch, Key Fob and Door Switch. The code doesn't save status to a data base, or forward it to anything else, it just joins the devices and watches them; you'll need to adapt it to what you want to do.
The piece that is still missing is support for an accelerometer that is inside the key fob. I don't have a clue how to initialize and use it. I don't need it, it would just be nice to understand. Maybe after I get some of the other things I'm working on done I'll come back and take another look.
Have fun.
Well, my partner in questionable activity in hacking the Iris Contact Switch and Key Fob has gotten his devices to work as well as mine, so it's time to close this project off for a while.
The reason my setup worked well and his didn't was because he didn't have a router on his network of Iris devices and I did. The Iris Smart Switch is a router and I have a handful of them scattered around the house. When he plugged one in and tried the devices, his started working and away he went. I just got word that he is switching his network over to completely local control.
Why is a router necessary? Frankly, I'm not sure at this point and I'll check into it more over time, but I have a couple of door switches working quite well, and the key fob controlling one of my smart switches. The system works nicely.
Between the two of us we managed to decode the various timers and such so you folk can pick up where we left off and implement these little devices in your own home for whatever you want. Nicely made (physically) product that was hampered only by the special code Iris put in them to force you to use their hub. I don't know why manufacturers insist on doing that, especially since there are folk like me that will take it as a challenge.
Here's the latest code with the various items. It will support the Iris smart switch, Key Fob and Door Switch. The code doesn't save status to a data base, or forward it to anything else, it just joins the devices and watches them; you'll need to adapt it to what you want to do.
#! /usr/bin/python ''' Hacking into the iris door sensor Have fun ''' from xbee import ZigBee import datetime import time import serial import sys, traceback import shlex import Queue from struct import * import binascii import inspect # line number for debugging def getLineNumber(): return inspect.stack()[1][2] # show data formatted so I can read it def showData(data): print "********** Message Contents" for key, value in data.iteritems(): if key == "id": print key, value else: print key, "".join("%02x " % ord(b) for b in data[key]) print "**********" def showClusterData(lAddr,sAddr, clusterId, data): print int(time.time()), print "".join("%02x" % ord(b) for b in lAddr) + \ " " + \ "".join("%02x" % ord(b) for b in sAddr) + \ " clid "+"%04x" % clusterId + "-" + \ "".join("%02x " % ord(b) for b in data) # this is a call back function for XBee receive. # When a message comes in this function will # get the data. # I had to use a queue to make sure there was enough time to # decode the incoming messages. Otherwise, in heavy traffic # periods, I'd get a new message while I was still working on # the last one. def messageReceived(data): #print "queueing message" messageQueue.put(data) def handleMessage(data): try: # if data['source_addr_long'] not in \ # ['\x00\x0d\x6f\x00\x04\x51\x07\x82',]: # return #print 'gotta packet' #showData(data) if (data['id'] == 'rx_explicit'): #print "RX Explicit" #showData(data) clusterId = (ord(data['cluster'][0])*256) + ord(data['cluster'][1]) #print 'Cluster ID:', hex(clusterId), if (data['profile']=='\x00\x00'): # The General Profile print 'Cluster ID:', hex(clusterId), print "profile id:", repr(data['profile']) if (clusterId == 0x0000): print ("Network (16-bit) Address Request") #showData(data) elif (clusterId == 0x0004): # Simple Descriptor Request, print("Simple Descriptor Request") #showData(data) elif (clusterId == 0x0005): # Active Endpoint Request, print("Active Endpoint Request") #showData(data) elif (clusterId == 0x0006): print "Match Descriptor Request" ''' the switch looks for clusters under profile c216, and I respond with only 1 cluster 02 ''' showData(data) time.sleep(2) print "Sending match descriptor response" zb.send('tx_explicit', dest_addr_long = data['source_addr_long'], dest_addr = data['source_addr'], src_endpoint = '\x00', dest_endpoint = '\x00', cluster = '\x80\x06', profile = '\x00\x00', options = '\x01', data = data['rf_data'][0:1] + '\x00\x00\x00\x01\x02' ) # The contact switch is a bit slow, give it # some time to digest the messages. time.sleep(2) zb.send('tx_explicit', dest_addr_long = data['source_addr_long'], dest_addr = data['source_addr'], src_endpoint = '\x02', dest_endpoint = '\x02', cluster = '\x00\xf6', profile = '\xc2\x16', data = '\x11\x01\xfc' ) time.sleep(2) elif (clusterId == 0x0008): # I couldn't find a definition for this print("This was probably sent to the wrong profile") elif (clusterId == 0x13): # This is the device announce message. print 'Device Announce Message' # this will tell me the address of the new thing # so I'm going to send an active endpoint request print 'Sending active endpoint request' epc = '\xaa'+data['source_addr'][1]+data['source_addr'][0] print "".join("%02x " % ord(b) for b in epc) zb.send('tx_explicit', dest_addr_long = data['source_addr_long'], dest_addr = data['source_addr'], src_endpoint = '\x00', dest_endpoint = '\x00', cluster = '\x00\x05', profile = '\x00\x00', options = '\x01', data = epc ) elif (clusterId == 0x8000): print("Network (16-bit) Address Response") #showData(data) elif (clusterId == 0x8005): # this is the Active Endpoint Response This message tells you # what the device can do, but it isn't constructed correctly to match # what the switch can do according to the spec. This is another # message that gets it's response after I receive the Match Descriptor print 'Active Endpoint Response' # elif (clusterId == 0x0006): elif (clusterId == 0x8038): print("Management Network Update Request"); else: print ("Unimplemented Cluster ID", hex(clusterId)) print elif (data['profile']=='\xc2\x16'): # Alertme Specific if (clusterId == 0xee): clusterCmd = ord(data['rf_data'][2]) status = '' if (clusterCmd == 0x80): if (ord(data['rf_data'][3]) & 0x01): status = "ON" else: status = "OFF" elif (clusterId == 0xef): clusterCmd = ord(data['rf_data'][2]) status = data['rf_data'] # cut down on typing if (clusterCmd == 0x81): usage = unpack('<H', status[3:5])[0] elif (clusterCmd == 0x82): usage = unpack('<L', status[3:7])[0] / 3600 upTime = unpack('<L', status[7:11])[0] #print ("%s Minute Stats: Usage, %d Watt Hours; Uptime, %d Seconds" %(name, usage/3600, upTime)) elif (clusterId == 0xf0): showClusterData(data['source_addr_long'],data['source_addr'],clusterId,data['rf_data']) # If the cluster cmd byte is 'xfb', it's a status if data['rf_data'][2] == '\xfb': status = data['rf_data'] # just to make typing easier if status[3] == '\x1f': print " Door Sensor", print str(float(unpack("<h", status[8:10])[0])\ / 100.0 * 1.8 + 32) + "F", if ord(status[-1]) & 0x01 == 1: print "reed switch open", else: print "reed switch closed", if ord(status[-1]) & 0x02 == 0: print "tamper switch open", else: print "tamper switch closed", elif status[3] == '\x1c': # Never found anything useful in this print "Power Switch", elif status[3] == '\x1d': print " Key Fob", print str(float(unpack("<h", status[8:10])[0])\ / 100.0 * 1.8 + 32) + "F", unpack('<I',status[4:8])[0] print 'Counter', unpack('<I',status[4:8])[0], elif status[3] == '\x1e': # This indicates a door sensor # with an invalid temperature reading # the other items are OK print " Door Sensor", print "Temperature invalid", if ord(status[-1]) & 0x01 == 1: print "reed switch open", else: print "reed switch closed", if ord(status[-1]) & 0x02 == 0: print "tamper switch open", else: print "tamper switch closed", #This may be the missing link to this thing print 'sending missing link', zb.send('tx_explicit', dest_addr_long = data['source_addr_long'], dest_addr = data['source_addr'], src_endpoint = data['dest_endpoint'], dest_endpoint = data['source_endpoint'], cluster = '\x00\xf0', profile = '\xc2\x16', data = '\x11\x39\xfd' ) pass else: print " Don't know this device yet", print '' else: print " Unknow cluster command" print '' pass elif (clusterId == 0x00f2): showClusterData(data['source_addr_long'],data['source_addr'],clusterId,data['rf_data']) print 'Tamper Switch Changed State to', status = data['rf_data'] if ord(status[3]) == 0x02: print "Open", else: print "Closed", print '' pass elif (clusterId == 0x00f3): showClusterData(data['source_addr_long'],data['source_addr'],clusterId,data['rf_data']) print ' Key Fob Button', status = data['rf_data'] print ord(status[3]), if status[2] == '\x01': print 'Closed', elif status[2] == '\x00': print 'Open', else: print 'Unknown', print 'Counter', unpack('<H',status[5:7])[0], print '' pass elif (clusterId == 0xf6): showClusterData(data['source_addr_long'],data['source_addr'],clusterId,data['rf_data']) print '' print "Identify Message" #extract vendor strings v = data['rf_data'] vendorstr = " - Vendor:" start = 21 datalen=len(v) while(start < datalen): slen=ord(v[start]) vendorstr = vendorstr + " " + v[start+1:start+1+slen] start = start+slen+1 print vendorstr print "Sending init message" zb.send('tx_explicit', dest_addr_long = data['source_addr_long'], dest_addr = data['source_addr'], src_endpoint = '\x00', dest_endpoint = '\x02', cluster = '\x00\xf0', profile = '\xc2\x16', data = '\x19\x41\xfa\x00\x01' ) elif (clusterId == 0x0500): # This is the security cluster showClusterData(data['source_addr_long'],data['source_addr'],clusterId,data['rf_data']) showData(data) # When the switch first connects, it come up in a state that needs # initialization, this command seems to take care of that. # So, look at the value of the data and send the command. if data['rf_data'][3:7] == '\x15\x00\x39\x10': print "sending initialization" zb.send('tx_explicit', dest_addr_long = data['source_addr_long'], dest_addr = data['source_addr'], src_endpoint = data['dest_endpoint'], dest_endpoint = data['source_endpoint'], cluster = '\x05\x00', profile = '\xc2\x16', data = '\x11\x80\x00\x00\x05' ) # The switch state is in byte [3] and is a bitfield # bit 0 is the magnetic reed switch state # bit 3 is the tamper switch state switchState = ord(data['rf_data'][3]) if switchState & 0x04: print 'Tamper Switch Closed', else: print 'Tamper Switch Open', if switchState & 0x01: print 'Reed Switch Opened', else: print 'Reed Switch Closed', print '' pass else: print ("Unimplemented Cluster ID", hex(clusterId)) print else: print ("Unimplemented Profile ID") elif(data['id'] == 'route_record_indicator'): print("Route Record Indicator") else: print("some other type of packet") print(data) except: print "I didn't expect this error:", sys.exc_info()[0] traceback.print_exc() def stopXBee(): print("XBee stop handler") zb.halt() ser.close() ####################### Actually Starts Here ################################ #------------ XBee Stuff ------------------------- # this is the /dev/serial/by-id device for the USB card that holds the XBee ZIGBEEPORT = "/dev/serial/by-id/usb-FTDI_FT232R_USB_UART_A901QL3F-if00-port0" ZIGBEEBAUD_RATE = 57600 # Open serial port for use by the XBee ser = serial.Serial(ZIGBEEPORT, ZIGBEEBAUD_RATE) # The XBee addresses I'm dealing with BROADCAST = '\x00\x00\x00\x00\x00\x00\xff\xff' UNKNOWN = '\xff\xfe' # This is the 'I don't know' 16 bit address # create a queue to put the messages into so they can # be handled in turn without one interrupting the next. messageQueue = Queue.Queue(0) # Create XBee library API object, which spawns a new thread zb = ZigBee(ser, callback=messageReceived) print "started" while True: try: if messageQueue.qsize() > 0: #print "getting message" message = messageQueue.get() handleMessage(message) messageQueue.task_done(); sys.stdout.flush() # if you're running non interactive, do this except KeyboardInterrupt: print "Keyboard interrupt" zb.halt() ser.close() break except: print "Unexpected error:", sys.exc_info()[0] traceback.print_exc() break print ("After the while") # just in case zb.halt() ser.close()Remember, for this to work properly, you'll need one of the Smart Switches in the network, but the Smart Switch that can measure power usage as well as control it was what got me into looking at these devices in the first place. Remember to look at the previous posts on this project as well, I may have forgotten to mention something.
The piece that is still missing is support for an accelerometer that is inside the key fob. I don't have a clue how to initialize and use it. I don't need it, it would just be nice to understand. Maybe after I get some of the other things I'm working on done I'll come back and take another look.
Have fun.
Friday, June 19, 2015
MQTT, Early Lessons Learned, and a Concern
So, I'm stepping through code trying to work out the best way of switching to mqtt for handling data from various devices and commands to control them and it DIED.
Yep, I couldn't publish anything even though everything was connected. Without anything being published, I naturally couldn't update the weather data base, and everything got lost. Losing data is no big deal, I'll get more; it's not having a clue what went wrong that sucks. I went and looked at the logging for mqtt and enabled it to syslog so I could at least see something, and that's when it started working again. Probably just a coincidence, but I'm leaving the broker logging enabled for a while. During this process I found a nice tool for interacting with mqtt from my Windows laptop mqtt-spy, it is a java application and allows me to stuff things into mqtt and monitor any of the 'topics', but all it told me was that nothing was getting published.
I'll just wait for it to happen again and see if I can get more information.
Otherwise, I decided to organize the topics differently. This is the first of what will probably be a large number of decisions like this ... mostly wrong. Instead of having the weather devices under the topic Desert-Home/Weather, they're now under Desert-Home/Device. I did this because I realized that weather is a collection of readings, not a device. So, the AcuRite 5n1 and the barometer are now Desert-Home/Device/5n1 and Desert-Home/Device/Barometer.
Yes, a comment on my last post by Grant got me to thinking.
As I was moving and changing code, I thought of something else; wouldn't it be cool to create a log of all the activity of the various things in one place on one machine so I could prowl through the day to day operation easily? Thus was invented mqttlogger.py. This new process subscribes to two topics: Desert-Home/Log and Desert-Home/Attention and simply writes what it receives to stdout. I redirect stdout to a file in the init configuration file and ta da, a log of whatever activity I want in one place. The topic Desert-Home/Attention is for stuff like the batteries going dead in the weather head or one of my battery operated devices. I'll eventually hook email to the Attention topic so I can be notified more easily.
The possible plan is to create a topic called Weather that has levels for the various readings that matter. There would be Desert-Home/Weather/CurrentTemp, HighestDaily, CurrentBarometer, RainfallToday, etc. Then I could pick and choose from the various values to present when I get that far. Frankly though, that's a lot of work and it may get modified to a JSON string that represents the weather reading right now. We'll see when I get there.
The weather station software has been converted to use mqtt now and is working reasonably well with the caveat that it may stop at any time because of the problem I mentioned up top. I can monitor the various things using mqtt-spy to see each device and the items involved including the log I created.
I'll post the code for the logger soon, I want to get a little time on it before I go and embarrass myself, and I want to give it a couple of days before cramming it into GitHub.
If it wasn't for that nagging possibility of it failing again I'd be really pleased.
Yep, I couldn't publish anything even though everything was connected. Without anything being published, I naturally couldn't update the weather data base, and everything got lost. Losing data is no big deal, I'll get more; it's not having a clue what went wrong that sucks. I went and looked at the logging for mqtt and enabled it to syslog so I could at least see something, and that's when it started working again. Probably just a coincidence, but I'm leaving the broker logging enabled for a while. During this process I found a nice tool for interacting with mqtt from my Windows laptop mqtt-spy, it is a java application and allows me to stuff things into mqtt and monitor any of the 'topics', but all it told me was that nothing was getting published.
I'll just wait for it to happen again and see if I can get more information.
Otherwise, I decided to organize the topics differently. This is the first of what will probably be a large number of decisions like this ... mostly wrong. Instead of having the weather devices under the topic Desert-Home/Weather, they're now under Desert-Home/Device. I did this because I realized that weather is a collection of readings, not a device. So, the AcuRite 5n1 and the barometer are now Desert-Home/Device/5n1 and Desert-Home/Device/Barometer.
Yes, a comment on my last post by Grant got me to thinking.
As I was moving and changing code, I thought of something else; wouldn't it be cool to create a log of all the activity of the various things in one place on one machine so I could prowl through the day to day operation easily? Thus was invented mqttlogger.py. This new process subscribes to two topics: Desert-Home/Log and Desert-Home/Attention and simply writes what it receives to stdout. I redirect stdout to a file in the init configuration file and ta da, a log of whatever activity I want in one place. The topic Desert-Home/Attention is for stuff like the batteries going dead in the weather head or one of my battery operated devices. I'll eventually hook email to the Attention topic so I can be notified more easily.
The possible plan is to create a topic called Weather that has levels for the various readings that matter. There would be Desert-Home/Weather/CurrentTemp, HighestDaily, CurrentBarometer, RainfallToday, etc. Then I could pick and choose from the various values to present when I get that far. Frankly though, that's a lot of work and it may get modified to a JSON string that represents the weather reading right now. We'll see when I get there.
The weather station software has been converted to use mqtt now and is working reasonably well with the caveat that it may stop at any time because of the problem I mentioned up top. I can monitor the various things using mqtt-spy to see each device and the items involved including the log I created.
I'll post the code for the logger soon, I want to get a little time on it before I go and embarrass myself, and I want to give it a couple of days before cramming it into GitHub.
If it wasn't for that nagging possibility of it failing again I'd be really pleased.
Thursday, June 18, 2015
MQTT Conversion, First Steps
Last post I talked about trying out mqtt and being impressed with the ease of using it. It really went well and I decided to convert my house monitoring system to using this tool. So I took the very first steps. This is going to be a complicated conversion because I already have a system running and I don't want to break it while I'm moving to something different, so I decided on a few things to get me started.
I'm going to divide the sensors, saving data, and presenting data into different things. For example, I'm starting with the weather station because it's the simplest, but still has multiple sensing devices. The entire house is published under the 'topic' Desert-Home, so the weather station will be Desert-home/Weather, and the two current sensors are the Acurite 5n1 and the barometer out on the fence. The interesting thing here is that they are radically different.
The 5n1 is detailed on about a hundred posts on this blog, but finally wound up being a sensor on the roof that transmits in the 933MHz range. I catch and decode the signal with an SDR (software defined radio) and some code I stole and heavily modified. The readings from this sensor will be published under Desert-Home/Weather/5n1.
The barometer is a device I built up myself and put in a Stevenson Screen out on a fence post in the yard. It has an XBee that sends a JSON string to a Pi. This will be published under Desert-Home/Weather/Barometer.
I can see each of these devices by simply subscribing to 'Desert-Home/Weather/#'. The pound sign means to get everything that comes in under 'Desert-Home/Weather' and that will include every sensor I come up with related to weather.
I also decided to use JSON strings as much as possible for a few reasons. Most important to me is the ease of reading a JSON string when you're monitoring the network to see why something is messing up. The JSON string has key:value pairs that tell you what it is and what the value is. This helps a lot when I'm looking for a problem. This will be painful since most of my sensors use another technique entirely and they will have to be converted. Another reason important to me is that I don't have to write parsers to get the data back into variables, There are libraries that will do that for me and all I have to do is use them. Maybe that will make up for the pain I go through converting the devices.
So, since the XBee network I have is on a different Pi than the weather station RF software, I get to try out a multi-machine solution as the very first thing I try. Step one is done; I brought up a mqtt server (mosquitto, see previous post) and added publish lines to the XBee receive code and the weather station decoding. It worked on the second try; I had a couple of syntax errors to fix.
Mosquitto caught the data and all I have to do to see it is use the tool mosquitto_sub to watch the data as it is generated. The command line:
mosquitto_sub -h "192.168.0.205" -t "Desert-Home/Weather/#" -v
shows me what is coming in as it happens:
The code is not in GitHub, I'm going to hold off until I get enough in there to make a significant difference. But, the example from the last post is exactly what I did with the difference above. This is a lot simpler and more versatile than I expected.
I'm going to divide the sensors, saving data, and presenting data into different things. For example, I'm starting with the weather station because it's the simplest, but still has multiple sensing devices. The entire house is published under the 'topic' Desert-Home, so the weather station will be Desert-home/Weather, and the two current sensors are the Acurite 5n1 and the barometer out on the fence. The interesting thing here is that they are radically different.
The 5n1 is detailed on about a hundred posts on this blog, but finally wound up being a sensor on the roof that transmits in the 933MHz range. I catch and decode the signal with an SDR (software defined radio) and some code I stole and heavily modified. The readings from this sensor will be published under Desert-Home/Weather/5n1.
The barometer is a device I built up myself and put in a Stevenson Screen out on a fence post in the yard. It has an XBee that sends a JSON string to a Pi. This will be published under Desert-Home/Weather/Barometer.
I can see each of these devices by simply subscribing to 'Desert-Home/Weather/#'. The pound sign means to get everything that comes in under 'Desert-Home/Weather' and that will include every sensor I come up with related to weather.
I also decided to use JSON strings as much as possible for a few reasons. Most important to me is the ease of reading a JSON string when you're monitoring the network to see why something is messing up. The JSON string has key:value pairs that tell you what it is and what the value is. This helps a lot when I'm looking for a problem. This will be painful since most of my sensors use another technique entirely and they will have to be converted. Another reason important to me is that I don't have to write parsers to get the data back into variables, There are libraries that will do that for me and all I have to do is use them. Maybe that will make up for the pain I go through converting the devices.
So, since the XBee network I have is on a different Pi than the weather station RF software, I get to try out a multi-machine solution as the very first thing I try. Step one is done; I brought up a mqtt server (mosquitto, see previous post) and added publish lines to the XBee receive code and the weather station decoding. It worked on the second try; I had a couple of syntax errors to fix.
Mosquitto caught the data and all I have to do to see it is use the tool mosquitto_sub to watch the data as it is generated. The command line:
mosquitto_sub -h "192.168.0.205" -t "Desert-Home/Weather/#" -v
shows me what is coming in as it happens:
Desert-Home/Weather/Barometer {"Barometer":{"temperature":"114.4","pressure":"1011.4","utime":"1434639446"}} Desert-Home/Weather/5n1 {"sensorId":{"SID":"92","t":"1434664650"},"channel":{"CH":"A","t":"1434664650"},"messageCaught":{"MC":"0","t":"1434664650"},"battLevel":{"BAT":"7","t":"1434664650"},"windSpeed":{"WS":"6.0","t":"1434664650"},"windDirection":{"WD":"W","t":"1434664650"},"temperature":{"T":"114.6","t":"1434664632"},"humidity":{"H":"9","t":"1434664632"},"rainCounter":{"RC":"445","t":"1434664650"}} Desert-Home/Weather/5n1 {"sensorId":{"SID":"92","t":"1434664650"},"channel":{"CH":"A","t":"1434664650"},"messageCaught":{"MC":"0","t":"1434664650"},"battLevel":{"BAT":"7","t":"1434664650"},"windSpeed":{"WS":"6.0","t":"1434664650"},"windDirection":{"WD":"W","t":"1434664650"},"temperature":{"T":"114.6","t":"1434664632"},"humidity":{"H":"9","t":"1434664632"},"rainCounter":{"RC":"445","t":"1434664650"}} Desert-Home/Weather/5n1 {"sensorId":{"SID":"92","t":"1434664650"},"channel":{"CH":"A","t":"1434664650"},"messageCaught":{"MC":"0","t":"1434664650"},"battLevel":{"BAT":"7","t":"1434664650"},"windSpeed":{"WS":"6.0","t":"1434664650"},"windDirection":{"WD":"W","t":"1434664650"},"temperature":{"T":"114.6","t":"1434664632"},"humidity":{"H":"9","t":"1434664632"},"rainCounter":{"RC":"445","t":"1434664650"}}
Yes, the outside temperature is REALLY 114.4, remember I live in Arizona and this is the middle of June, but it's a dry heat, the humidity reading is 9%. As if that matters when the temperature is this high ...
But, also notice that I have both strings coming to the subscription even though they are very different devices, on different Pi's. Heck, they could be half way around the world and it wouldn't matter, it would still work. I tested that with a conspirator just this morning.
Now to follow my plans, I have to catch this data in some other process and save it to my database. I'll get to that soon, maybe even today, but I'm going to hold off on changing the presentation to the web for later because I have many different presentations to consider when I get to that point. See, currently I take everything presented from my data bases. If I'm using my Android app to look at the house, the data is coming out of the data base. When I send a command to close the garage doors, the command is executed and the app doesn't update until it is recorded in the data base. I may change that to reflect the latest reading from the mqtt server. We'll see, and then I'll probably change it.
The code is not in GitHub, I'm going to hold off until I get enough in there to make a significant difference. But, the example from the last post is exactly what I did with the difference above. This is a lot simpler and more versatile than I expected.
Wednesday, June 17, 2015
OK, Fine, I'll take a look at MQTT
A number of my readers have mentioned mqtt as a great tool for the kind of thing I'm doing around the house. A couple of them have pointed out that what I'm doing could be done much easier using mqtt and even sent me web sites to look at.
Naturally, I resisted. That's sort of the way I am, but also, the various web sites out there make it look so complex that I just didn't want to try and deal with it. Another stinking process with an API and things I had to learn to use it ... gag!
Then Glenn sent me a link to an Arduino implementation of publishing data to an mqtt server <link> and things clicked into place. What happened was the Arduino was too small and slow for fancy implementations, it had to fit in a couple of K and run on a 16MHz machine, so it was as simple as it could be. That got me to thinking --- there's really no protocol, you build it yourself, There's no registration, special language, ... none of the stuff that drives me nuts. You just use it.
Fine, I've eaten my words before, so I gave it a shot. AWESOME set of tools. This thing is going to be the center piece of a major update to my house control system.
OK, Glen, Brant, others too many to itemize, go ahead ... scream, "I told you so, but you wouldn't listen." I deserve it.
The way I started was to install the mqtt client software on my weather station Pi. I put a few lines in the python code to import, initialize an object and then a single line to send the data to a public mqtt server. Yes, that's literally all there was to publishing data, one single line of actual code.
Then I used the one of the tools to see if it actually got there and I could read it back.
If you've been following the weather station posts, these lines will look really familiar. The tool mosqitto_sub is a test subscriber that took the public server I used, 'test.mosquitto.org', and looked for the published data keyed under 'Desert-Home/Weather/String' and printed it to the console of the Pi. So, what was happening was that my weather station Pi was reading the data from a radio and sending it over the internet to a server. Then on a different Pi, I was reading the data back as it came in from the server I subscribed to. Three machines involved working over the internet and all I had to do was write one line of actual code to do it.
To try it out, install the mqtt client software for python and the command line tool I used above. No, I didn't misspell it, there really are two 't's in there.
sudo apt-get install mosquitto mosquitto-clients
sudo pip install paho-mqtt
This will actually install the mqtt broker and start it up, but you can stop it by the command:
sudo /etc/init.d/mosquitto stop
Then, in some code somewhere, put the lines:
import paho.mqtt.client as mqtt
The string "Desert-Home/Weather/String" is called a 'topic'. This is a hierarchical set of keys that you construct for your own purposes. I chose "Desert-Home" for obvious reasons, then "Weather" because it's for the weather station, then "String" because I was publishing the JSON string produced by my code that reads the AcuRite 5n1 weather head. I may create another item "OutsideTemperature" to hold the latest outside reading from the fence post sensor in which case it would be:
"Desert-Home/Weather/OutsideTemperature."
Then to get the data back as it is published to the server, use a line like:
mosquitto_sub -h "test.mosquitto.org" -t "Whatever/you/decided" -v
Slick isn't it. There's no mystery here, no special stuff you have to learn, it's all totally up to you how you use it. To make it more versatile, the '#' and '+' characters have special meaning that can help you debug and select things. Check the various sites out there for the usage of those symbols.
There's a ba-jillion web sites out there that talk about mqtt, so I'm not going to recap the light-weight, short transaction, fail safe blather that they cover ad nauseum, instead over the next few weeks, I'm going to convert my house monitor and control system to actually use it and try to show you how to follow my footsteps improving and changing it to suit your particular needs. I'm sure I'm going to mess it up and have to redo things a couple of hundred times, but this should, at least, be good for a laugh.
During this process I'm going to visit my sensor devices and update the code on them to my latest experience level and implement several changes to them as well. JSON strings as data output and input so I can get rid of parsers, that kind of stuff. I'll gateway the devices from the XBee network to messages that will be published to mqtt and caught by whatever needs to handle the data. I don't know what I'm going to do with the Iris devices yet, but something will come to me.
Stay tuned.
Naturally, I resisted. That's sort of the way I am, but also, the various web sites out there make it look so complex that I just didn't want to try and deal with it. Another stinking process with an API and things I had to learn to use it ... gag!
Then Glenn sent me a link to an Arduino implementation of publishing data to an mqtt server <link> and things clicked into place. What happened was the Arduino was too small and slow for fancy implementations, it had to fit in a couple of K and run on a 16MHz machine, so it was as simple as it could be. That got me to thinking --- there's really no protocol, you build it yourself, There's no registration, special language, ... none of the stuff that drives me nuts. You just use it.
Fine, I've eaten my words before, so I gave it a shot. AWESOME set of tools. This thing is going to be the center piece of a major update to my house control system.
OK, Glen, Brant, others too many to itemize, go ahead ... scream, "I told you so, but you wouldn't listen." I deserve it.
The way I started was to install the mqtt client software on my weather station Pi. I put a few lines in the python code to import, initialize an object and then a single line to send the data to a public mqtt server. Yes, that's literally all there was to publishing data, one single line of actual code.
Then I used the one of the tools to see if it actually got there and I could read it back.
pi@deserthome:~/src/other-things$ mosquitto_sub -h "test.mosquitto.org" -t "Desert-Home/#" -v Desert-Home/Weather/String {"sensorId":{"SID":"92","t":"1434572606"},"channel":{"CH":"A","t":"1434572606"},"messageCaught":{"MC":"0","t":"1434572606"},"battLevel":{"BAT":"7","t":"1434572606"},"windSpeed":{"WS":"4.0","t":"1434572606"},"windDirection":{"WD":"SSE","t":"1434572587"},"temperature":{"T":"111.7","t":"1434572606"},"humidity":{"H":"11","t":"1434572606"},"rainCounter":{"RC":"445","t":"1434572587"}} Desert-Home/Weather/String {"sensorId":{"SID":"92","t":"1434572606"},"channel":{"CH":"A","t":"1434572606"},"messageCaught":{"MC":"0","t":"1434572606"},"battLevel":{"BAT":"7","t":"1434572606"},"windSpeed":{"WS":"4.0","t":"1434572606"},"windDirection":{"WD":"SSE","t":"1434572587"},"temperature":{"T":"111.7","t":"1434572606"},"humidity":{"H":"11","t":"1434572606"},"rainCounter":{"RC":"445","t":"1434572587"}} Desert-Home/Weather/String {"sensorId":{"SID":"92","t":"1434572623"},"channel":{"CH":"A","t":"1434572623"},"messageCaught":{"MC":"0","t":"1434572623"},"battLevel":{"BAT":"7","t":"1434572623"},"windSpeed":{"WS":"2.5","t":"1434572623"},"windDirection":{"WD":"SSW","t":"1434572623"},"temperature":{"T":"111.7","t":"1434572606"},"humidity":{"H":"11","t":"1434572606"},"rainCounter":{"RC":"445","t":"1434572623"}}
To try it out, install the mqtt client software for python and the command line tool I used above. No, I didn't misspell it, there really are two 't's in there.
sudo apt-get install mosquitto mosquitto-clients
sudo pip install paho-mqtt
This will actually install the mqtt broker and start it up, but you can stop it by the command:
sudo /etc/init.d/mosquitto stop
Then, in some code somewhere, put the lines:
import paho.mqtt.client as mqtt
mqttc = mqtt.Client()
mqttc.connect("test.mosquitto.org", 1883, 60)
mqttc.loop_start()
To create the object, initialize it and start it up. Then the single line somewhere you are logging or recording something like this:
mqttc.publish("Desert-Home/Weather/String",buff);
"Desert-Home/Weather/OutsideTemperature."
Then to get the data back as it is published to the server, use a line like:
mosquitto_sub -h "test.mosquitto.org" -t "Whatever/you/decided" -v
Slick isn't it. There's no mystery here, no special stuff you have to learn, it's all totally up to you how you use it. To make it more versatile, the '#' and '+' characters have special meaning that can help you debug and select things. Check the various sites out there for the usage of those symbols.
There's a ba-jillion web sites out there that talk about mqtt, so I'm not going to recap the light-weight, short transaction, fail safe blather that they cover ad nauseum, instead over the next few weeks, I'm going to convert my house monitor and control system to actually use it and try to show you how to follow my footsteps improving and changing it to suit your particular needs. I'm sure I'm going to mess it up and have to redo things a couple of hundred times, but this should, at least, be good for a laugh.
During this process I'm going to visit my sensor devices and update the code on them to my latest experience level and implement several changes to them as well. JSON strings as data output and input so I can get rid of parsers, that kind of stuff. I'll gateway the devices from the XBee network to messages that will be published to mqtt and caught by whatever needs to handle the data. I don't know what I'm going to do with the Iris devices yet, but something will come to me.
Stay tuned.
Friday, May 29, 2015
Arduino Controlled Sprinklers, and A Guest Speaker
Yes, I have a guest speaker for this post. Glenn lives in the part of Texas where they got about a ton (per square inch) of rain recently. Since he couldn't play outside, he decided to play inside with ... yes, a sprinkler system. I asked him to write about it because it is a novel and really, really cool setup. It uses those valves that latch into place and take zero current to run saving power and allowing for the possibility of battery operation. I'll shut up now and let Glenn take over:
Lately I have been working on my baler between rain storms. However, during the rain storms I have continued to work on my vineyard irrigation controller. Not that the grapes are going to need watering any time soon. But there will come a time and I must get this moved forward.
I have a large aluminum panel mounted to my steel pipe fence with a water hydrant next to it. I have my three zones of irrigation pipes running up to this panel (just underneath it). I have purchased a three zone manifold valve system from Orbit and equipped it with two 24VDC solenoids. The black object on top of the green valves is the solenoid. The 24VAC ones have two black wires. The 24VDC solenoids have one red and one black wire. The red goes to the set screw terminal and the black goes to the reset terminal of the zone. This is a really easy thing to do. The manifold will be mounted on the lower left of the panel just above the zone pipes. There is no need for me to put these in the ground as it is more of an industrial installation. The aluminum panel faces north so nothing will get direct sunlight, which here in East Texas is intense. Currently the aluminium panel has just a water filter mounted on it. I welded Unistrut ( this is the funny C channel electricians use) to the pipe fence then bolted the panel to the Unistrut. It makes taking the whole panel off really easy should it be necessary.
Back to the controller. Just above the valves I will be mounting a 4X4X2 inch plastic electrical box. I got this box from Home Depot. I comes with a waterproof cover and mounting tabs on each side. This is a shot of it open with the cover reattached backwards.
As you can see from the above photo I will be mounting my ardweeny and support shields to the cover of the box, not inside the box. There will be one or two access holes in the bottom with water tight entry fittings through which the wires will got to the valves as shown in the first picture. There will be three shields to the system. Ardweeny, XBee and Valve controller, all stacked as shown.
The really key thing here is that everything will mount to the cover, not inside the box.
I have several reasons for this. All fall under the category of convenience.
When I open the box and reverse the cover as shown, everything is readily available. I can access the usb port without problem. I can remove any or all of the boards conveniently. I can work with the enclosed wiring without having the shields also in the box getting in the way.
If I need to I can remove the entire controller just by disconnecting a few wires and take it away. The entire box stays mounted in place. Oh and by the way, since there are no screws on the back the box will site flat on the surface of the aluminium plate.
The box cover will also have a key pad mounted on the front for manually turning the irrigation zones off and on:
The Orbit manifold I purchased from Lowe's. They usually have them in stock. They come with everything you need including 24V AC solenoids. I think I paid $54.00 for it. I bought two 24V DC solenoids off Amazon that were "used" actually only the packaging was used for $7.00 each. The third I will buy from Lowe's locally as they are in stock and only cost about $10.00. They simply unscrew. My 24V DC solenoids came with a little gray adapter that has to be removed and then screw it into the manifold. I can take some pix and send them if you like to describe it.
Rain protection. None required. This stuff is meant to be in the ground. It gets wet. Everything will be mounted on a vertical plate of aluminium. If one wanted you could simply add a little roof shelter at the top of the aluminium, but I do not plan to. The 4X4X2 box is water tight. The connections coming in from the bottom mean rain can't enter unless it rains uphill.
I mentioned water proof connectors on the bottom will also prevent wasps, and other critters from getting at the circuitry.
I may put a little roof cover above the keypad like they do for gate openers that you drive up to. I will seal any entry points such as the keyboard cable entry with silicon sealant.
I plan on powering this with either a 9volt battery or a combination of 9volt and three AA's. This way I won't need to have a regulator. I plan on having it shut down through software much the same way as your remote temperature sensor works. The jury is still out just yet on this. I plan to liberally and unashamedly steel Dave's software......Hee Hee Hee. He's done a great job and created a voluminous library of functions that will save me from reinventing the wheel.
I chose to go with Ardweeny because there is a nice little shield for it that fits the Arduino form factor. I can do all my testing and code development with the regular Arduino and then simply replace it with the programmed Ardweeny. No other special mouning requirements. I have the shield on order and as soon as it arrives I can drill the holes.
I plan on a next issue of this showing how I will use insert nuts that have a built in standoff of about a 1/4 of an inch to mount the Ardweeny shield to the cover and gluing up of the finished numeric pad. There will also be more discussion on powering. I am thinking of a second box that could house a small motorcycle battery. I found one for $25.00. The big thing on this to remember is that this only will run through the summer months so I could conceivably charge the motorcycle battery once a year and run it all summer. More to come on this note.
Still a few things to think about with this yet. I'm not sure about a led in the outside of the cover plate that will illuminate when you push the button. I'm thinking that the clicking sound of the solenoid plus the rush sound of the water in the pipes might just be enough. Needs to pass the "wife" test though.
So there you have it. The main theme of the blog is really "Don't forget the cover" and the benefits it affords for mounting options.
Comments?
This is Dave again, I have a comment, AWESOME. I've been using those waterproof boxes from Home Depot for quite a while for various projects and it never, ever occurred to me to mount everything to the lid <link>. Simple idea that I should have thought of.
Yes, I'll be stealing that idea and others from him.
Lately I have been working on my baler between rain storms. However, during the rain storms I have continued to work on my vineyard irrigation controller. Not that the grapes are going to need watering any time soon. But there will come a time and I must get this moved forward.
I have a large aluminum panel mounted to my steel pipe fence with a water hydrant next to it. I have my three zones of irrigation pipes running up to this panel (just underneath it). I have purchased a three zone manifold valve system from Orbit and equipped it with two 24VDC solenoids. The black object on top of the green valves is the solenoid. The 24VAC ones have two black wires. The 24VDC solenoids have one red and one black wire. The red goes to the set screw terminal and the black goes to the reset terminal of the zone. This is a really easy thing to do. The manifold will be mounted on the lower left of the panel just above the zone pipes. There is no need for me to put these in the ground as it is more of an industrial installation. The aluminum panel faces north so nothing will get direct sunlight, which here in East Texas is intense. Currently the aluminium panel has just a water filter mounted on it. I welded Unistrut ( this is the funny C channel electricians use) to the pipe fence then bolted the panel to the Unistrut. It makes taking the whole panel off really easy should it be necessary.
Back to the controller. Just above the valves I will be mounting a 4X4X2 inch plastic electrical box. I got this box from Home Depot. I comes with a waterproof cover and mounting tabs on each side. This is a shot of it open with the cover reattached backwards.
As you can see from the above photo I will be mounting my ardweeny and support shields to the cover of the box, not inside the box. There will be one or two access holes in the bottom with water tight entry fittings through which the wires will got to the valves as shown in the first picture. There will be three shields to the system. Ardweeny, XBee and Valve controller, all stacked as shown.
The really key thing here is that everything will mount to the cover, not inside the box.
I have several reasons for this. All fall under the category of convenience.
When I open the box and reverse the cover as shown, everything is readily available. I can access the usb port without problem. I can remove any or all of the boards conveniently. I can work with the enclosed wiring without having the shields also in the box getting in the way.
If I need to I can remove the entire controller just by disconnecting a few wires and take it away. The entire box stays mounted in place. Oh and by the way, since there are no screws on the back the box will site flat on the surface of the aluminium plate.
The box cover will also have a key pad mounted on the front for manually turning the irrigation zones off and on:
The Orbit manifold I purchased from Lowe's. They usually have them in stock. They come with everything you need including 24V AC solenoids. I think I paid $54.00 for it. I bought two 24V DC solenoids off Amazon that were "used" actually only the packaging was used for $7.00 each. The third I will buy from Lowe's locally as they are in stock and only cost about $10.00. They simply unscrew. My 24V DC solenoids came with a little gray adapter that has to be removed and then screw it into the manifold. I can take some pix and send them if you like to describe it.
Rain protection. None required. This stuff is meant to be in the ground. It gets wet. Everything will be mounted on a vertical plate of aluminium. If one wanted you could simply add a little roof shelter at the top of the aluminium, but I do not plan to. The 4X4X2 box is water tight. The connections coming in from the bottom mean rain can't enter unless it rains uphill.
I mentioned water proof connectors on the bottom will also prevent wasps, and other critters from getting at the circuitry.
I may put a little roof cover above the keypad like they do for gate openers that you drive up to. I will seal any entry points such as the keyboard cable entry with silicon sealant.
I plan on powering this with either a 9volt battery or a combination of 9volt and three AA's. This way I won't need to have a regulator. I plan on having it shut down through software much the same way as your remote temperature sensor works. The jury is still out just yet on this. I plan to liberally and unashamedly steel Dave's software......Hee Hee Hee. He's done a great job and created a voluminous library of functions that will save me from reinventing the wheel.
I chose to go with Ardweeny because there is a nice little shield for it that fits the Arduino form factor. I can do all my testing and code development with the regular Arduino and then simply replace it with the programmed Ardweeny. No other special mouning requirements. I have the shield on order and as soon as it arrives I can drill the holes.
I plan on a next issue of this showing how I will use insert nuts that have a built in standoff of about a 1/4 of an inch to mount the Ardweeny shield to the cover and gluing up of the finished numeric pad. There will also be more discussion on powering. I am thinking of a second box that could house a small motorcycle battery. I found one for $25.00. The big thing on this to remember is that this only will run through the summer months so I could conceivably charge the motorcycle battery once a year and run it all summer. More to come on this note.
Still a few things to think about with this yet. I'm not sure about a led in the outside of the cover plate that will illuminate when you push the button. I'm thinking that the clicking sound of the solenoid plus the rush sound of the water in the pipes might just be enough. Needs to pass the "wife" test though.
So there you have it. The main theme of the blog is really "Don't forget the cover" and the benefits it affords for mounting options.
Comments?
This is Dave again, I have a comment, AWESOME. I've been using those waterproof boxes from Home Depot for quite a while for various projects and it never, ever occurred to me to mount everything to the lid <link>. Simple idea that I should have thought of.
Yes, I'll be stealing that idea and others from him.
Tuesday, May 26, 2015
Hacking Into The Iris Door Sensor, Part 3, The code
The previous post in the project is here <link>
Last time I described the Key Fob that Lowe's sells and previous to that I showed the Door Switch, but I haven't shown you the way I got them to work. I'm still a little reluctant because my partner in this project hasn't succeeded in getting his devices to work properly, but it occurred to me that someone out there might pick up the code and give it a try. I know several folk have used my Smart Switch code to control and monitor things around the house, maybe they're just waiting for me to publish how to do it.
At any rate, this is how I got the switches and key fob to behave:
I have eight devices, five of the smart switches, two door switches, and a key fob, and all of them are working with the same monitoring code. The code is only to monitor the devices, there's no provisions for control, Actually, the only thing you can control is the smart switches, and I've already posted code for that, so this will show you how to monitor all the devices. This code works pretty easily, pull the battery out of the device, start the code, put the battery back in and push the button eight times quickly. The switch will contact the code and join on its own. Once joined, the door switch and key fob will send status every two minutes telling you they're alive and in range. The smart switches send status much more often with a cumulative status every minute. If you want to join other devices, just pull the battery, put it back, and push the button a bunch of times; joining is always enabled and the next device should work fine. Stopping the code (cntrl-C) and restarting it won't cause the devices to leave the network, they'll be fine when you restart the code. I did this a LOT when I was trying to get them to work.
I could go into intimate details of how the interaction works, but the code will show you this much better. What happens is that the switch sends an device announce message and the code will respond and then they exchange data. During this there are two specific message that are sent to Alertme devices to get them to accept the XBee coordinator; this is the secret part of the Iris system that makes these devices unique. The code I came up with for the smart switches wouldn't work because the door switch and key fob are slower devices and I was hitting them too quickly with the initialization messages. They didn't have time to recognize and react to them. A couple of sleep calls took care of that problem. I used the same setup on the XBee as when I hacked into the Smart Switch, you can look at it here <link>
Once running, you can press a switch on the key fob and it will transmit a messsage telling you that the button was pressed or released along with a counter that represents the time in milliseconds off the action. By subtracting the released time from the pressed time you can look for a long press as opposed to a short press. Nice feature.
The door switch has two switches, a tamper switch that will tell you if the cover is taken off and a reed switch that tells you the door has been opened. The tamper switch will tell you press and release like the key fob does, as will the reed switch. This means you can tell if the door is opened or closed as well as the cover off or on.
Like the smart switches, the key fob and door switch once joined, stay that way. You can pull the battery and put it back in without it having a problem. You can also drive away out of range and come back and everything works just fine. If you kill the coordinator by pulling the power or something, you might have to make them rejoin. This all depends on how long it was off and what happened during the off period. Sometimes the coordinator XBee will choose a different channel and then the switches will have trouble finding it. A simple power failure won't cause any problem, but reloading software to the controller could mean a visit to the devices to force them to rejoin. This kind of problem can be avoided by using the parameters on the XBee controller to limit it to a single predetermined channel. Doing this means the XBee comes up, establishes a network on the channel and the devices then just continue on as before.
So, you're going to get a lot of data as you add devices. Each device will interact at least every couple of minutes and every time you mess with them. For me, this meant missed messages. The traffic was pretty steady, but things didn't look right until I added a queue to the code to hold the messages and took them off one at a time to deal with. There's plenty of unused time to handle the messages, but they tend to come in in bunches and can cause problems. Adding a queue seemed to help that problem a lot; I don't see any missing or partial messages when I run it.
There's lots of debug and logging in the code as well as a test to help you isolate a single device. You'll see it in the code; I look for a list of devices to listen to and ignore the others. I had to do this to keep the other devices on the network from confusing me when I was trying figure out which bit meant what. Look for the test at the top of the code to decode the messages and adjust it as you need to. Also, this doesn't save anything from one run to the next. There's no database of devices that gets updated and the state of the devices isn't saved to compare with later. Once you're comfortable with the code, simply add whatever you want to handle things. I'm going to hook this code in with my database of house devices to save the states and decide exactly what I want to do with it later when I have a little more experience.
Here's the code:
Yes, it's a bit complex. More complex than other hacks I've posted. I had to do this to allow for multiple devices that had multiple endpoints and clusters involved. It still isn't too hard to understand, and you should be fine when you start messing with it.
You'll notice that I keep a list of devices that have contacted the code. This is to allow you to tell what happened with various devices as the interaction continues. I log the long address, time and relevant details as they happen to help you understand how to deal with the incoming data. When you have several devices, the log can become a bit overpowering since nothing stops sending, it just goes on forever. Here's a bit of the logging so you can get an idea what it looks like:
The first field is the Unix timestamp. Basically the number of seconds since 1970. The second is the long address of the device; this never changes. Then the cluster id followed by the data specific to the cluster. I decoded the times for the key fob and the temperature for the door switch because those could be fun to play with. Notice that the smart switches really send a lot of data, this will help you understand why I limit the number of devices with a test at the top of the code. If everything is there, you have a heck of a time picking out what you want to see. The list of know devices is the long address and the currently assigned short address. The short address will change from time to time as you restart the process or pull a battery somewhere.
Remember, this runs on a Raspberry Pi, I haven't tried it on anything else. I have my XBee plugged into a USB port on the Pi, if you're using the serial port, just change the port in the code, that should be all that is needed. If you don't know which usb port to use, there's a ton of articles on the web that will help you find the port, just look around a bit, or to see how I did it look here <link>. What I'm hoping for is a couple of folk that want to use these nicely priced and well made devices with their own code for some project to grab it and give me some feedback on the interaction. It would be nice to see someone else running this code.
Have fun.
The next part to this project is here <link>
Last time I described the Key Fob that Lowe's sells and previous to that I showed the Door Switch, but I haven't shown you the way I got them to work. I'm still a little reluctant because my partner in this project hasn't succeeded in getting his devices to work properly, but it occurred to me that someone out there might pick up the code and give it a try. I know several folk have used my Smart Switch code to control and monitor things around the house, maybe they're just waiting for me to publish how to do it.
At any rate, this is how I got the switches and key fob to behave:
I have eight devices, five of the smart switches, two door switches, and a key fob, and all of them are working with the same monitoring code. The code is only to monitor the devices, there's no provisions for control, Actually, the only thing you can control is the smart switches, and I've already posted code for that, so this will show you how to monitor all the devices. This code works pretty easily, pull the battery out of the device, start the code, put the battery back in and push the button eight times quickly. The switch will contact the code and join on its own. Once joined, the door switch and key fob will send status every two minutes telling you they're alive and in range. The smart switches send status much more often with a cumulative status every minute. If you want to join other devices, just pull the battery, put it back, and push the button a bunch of times; joining is always enabled and the next device should work fine. Stopping the code (cntrl-C) and restarting it won't cause the devices to leave the network, they'll be fine when you restart the code. I did this a LOT when I was trying to get them to work.
I could go into intimate details of how the interaction works, but the code will show you this much better. What happens is that the switch sends an device announce message and the code will respond and then they exchange data. During this there are two specific message that are sent to Alertme devices to get them to accept the XBee coordinator; this is the secret part of the Iris system that makes these devices unique. The code I came up with for the smart switches wouldn't work because the door switch and key fob are slower devices and I was hitting them too quickly with the initialization messages. They didn't have time to recognize and react to them. A couple of sleep calls took care of that problem. I used the same setup on the XBee as when I hacked into the Smart Switch, you can look at it here <link>
Once running, you can press a switch on the key fob and it will transmit a messsage telling you that the button was pressed or released along with a counter that represents the time in milliseconds off the action. By subtracting the released time from the pressed time you can look for a long press as opposed to a short press. Nice feature.
The door switch has two switches, a tamper switch that will tell you if the cover is taken off and a reed switch that tells you the door has been opened. The tamper switch will tell you press and release like the key fob does, as will the reed switch. This means you can tell if the door is opened or closed as well as the cover off or on.
Like the smart switches, the key fob and door switch once joined, stay that way. You can pull the battery and put it back in without it having a problem. You can also drive away out of range and come back and everything works just fine. If you kill the coordinator by pulling the power or something, you might have to make them rejoin. This all depends on how long it was off and what happened during the off period. Sometimes the coordinator XBee will choose a different channel and then the switches will have trouble finding it. A simple power failure won't cause any problem, but reloading software to the controller could mean a visit to the devices to force them to rejoin. This kind of problem can be avoided by using the parameters on the XBee controller to limit it to a single predetermined channel. Doing this means the XBee comes up, establishes a network on the channel and the devices then just continue on as before.
So, you're going to get a lot of data as you add devices. Each device will interact at least every couple of minutes and every time you mess with them. For me, this meant missed messages. The traffic was pretty steady, but things didn't look right until I added a queue to the code to hold the messages and took them off one at a time to deal with. There's plenty of unused time to handle the messages, but they tend to come in in bunches and can cause problems. Adding a queue seemed to help that problem a lot; I don't see any missing or partial messages when I run it.
There's lots of debug and logging in the code as well as a test to help you isolate a single device. You'll see it in the code; I look for a list of devices to listen to and ignore the others. I had to do this to keep the other devices on the network from confusing me when I was trying figure out which bit meant what. Look for the test at the top of the code to decode the messages and adjust it as you need to. Also, this doesn't save anything from one run to the next. There's no database of devices that gets updated and the state of the devices isn't saved to compare with later. Once you're comfortable with the code, simply add whatever you want to handle things. I'm going to hook this code in with my database of house devices to save the states and decide exactly what I want to do with it later when I have a little more experience.
Here's the code:
#! /usr/bin/python ''' This is a hack into the operation of the Iris Smart Home Smart Plug, Door Switch, and Key Fob. The evolution of this is discussed on Desert-Home.com in detail. Have fun ''' from apscheduler.schedulers.background import BackgroundScheduler from xbee import ZigBee import logging import datetime import Queue import time import serial import sys, traceback import shlex from struct import * def printData(data): print "********** Message Contents" for key, value in data.iteritems(): if key == "id": print key, value else: print key, "".join("%02x " % ord(b) for b in data[key]) print "**********" def clusterData(lAddr,clusterId, data): print int(time.time()), print "".join("%02x" % ord(b) for b in lAddr) + \ " clid "+"%04x" % clusterId + "-" + \ "".join("%02x " % ord(b) for b in data) # this is a call back function. When a message # comes in this function will get the data # I had to use a queue to make sure there was enough time to # decode the incoming messages. Otherwise, in heavy traffic # periods, I'd get a new message while I was still working on # the last one. def messageReceived(data): #print "queueing message" messageQueue.put(data) def handleMessage (data): try: ''' I have a network of these devices and had to add this test to keep the log down to a reasonable size. If all the devices show up, it's hard to tell what is going on. Just uncomment the check and put whatever you want to watch in. ''' '''if data['source_addr_long'] not in \ ['\x00\x0d\x6f\x00\x03\xc2\x71\xcc', '\x00\x0d\x6f\x00\x02\x83\xfa\x4e']: return''' #print '' #print 'gotta packet', #printData(data) if (data['id'] == 'rx_explicit'): #print "RX Explicit" #printData(data) clusterId = (ord(data['cluster'][0])*256) + ord(data['cluster'][1]) #print 'Cluster ID:', hex(clusterId), #print "profile id:", repr(data['profile']) if (data['profile']=='\x00\x00'): # The General Profile print 'Cluster ID:', hex(clusterId), print "profile id:", repr(data['profile']) if (clusterId == 0x0000): print ("Network (16-bit) Address Request") #printData(data) elif (clusterId == 0x0004): # Simple Descriptor Request, print("Simple Descriptor Request") #printData(data) elif (clusterId == 0x0005): # Active Endpoint Request, print("Active Endpoint Request") #printData(data) elif (clusterId == 0x0006): print "Match Descriptor Request" #printData(data) time.sleep(2) print "Sending match descriptor response" zb.send('tx_explicit', dest_addr_long = data['source_addr_long'], dest_addr = data['source_addr'], src_endpoint = '\x00', dest_endpoint = '\x00', cluster = '\x80\x06', profile = '\x00\x00', options = '\x01', data = '\x04\x00\x00\x00\x01\x02' ) # The contact switch is a bit slow, give it # some time to digest the messages. time.sleep(2) zb.send('tx_explicit', dest_addr_long = data['source_addr_long'], dest_addr = data['source_addr'], src_endpoint = '\x02', dest_endpoint = '\x02', cluster = '\x00\xf6', profile = '\xc2\x16', data = '\x11\x01\xfc' ) time.sleep(2) elif (clusterId == 0x0008): # I couldn't find a definition for this print("This was probably sent to the wrong profile") elif (clusterId == 0x0013): # This is the device announce message. print 'Device Announce Message' # this will tell me the address of the new thing # so I'm going to send an active endpoint request print 'Sending active endpoint request' epc = '\xaa'+data['source_addr'][1]+data['source_addr'][0] print "".join("%02x " % ord(b) for b in epc) zb.send('tx_explicit', dest_addr_long = data['source_addr_long'], dest_addr = data['source_addr'], src_endpoint = '\x00', dest_endpoint = '\x00', cluster = '\x00\x05', profile = '\x00\x00', options = '\x01', data = epc ) #printData(data) elif (clusterId == 0x8000): print("Network (16-bit) Address Response") #printData(data) elif (clusterId == 0x8038): print("Management Network Update Request"); elif (clusterId == 0x8005): # this is the Active Endpoint Response This message tells you # what the device can do print 'Active Endpoint Response' printData(data) elif (clusterId == 0x8004): print "simple descriptor response" else: print ("Unimplemented Cluster ID", hex(clusterId)) print elif (data['profile']=='\xc2\x16'): # Alertme Specific if data['source_addr_long'] not in devices: devices.setdefault(data['source_addr_long'], []).append(data['source_addr']) # suppress printing for known clusters # so I can look at it more closely if clusterId not in [0X0500, 0x00ef, 0x00f0, 0x00f2, 0x00f3,0x00f6]: printData(data) print "Unhandled Message" if (clusterId == 0xef): clusterData(data['source_addr_long'],clusterId,data['rf_data']) clusterCmd = ord(data['rf_data'][2]) status = data['rf_data'] # cut down on typing if (clusterCmd == 0x81): usage = unpack('<H', status[3:5])[0] print " Current Usage:", usage elif (clusterCmd == 0x82): usage = unpack('<L', status[3:7])[0] / 3600 upTime = unpack('<L', status[7:11])[0] print (" Switch Minute Stats: Usage, %d Watt Hours; Uptime, %d Seconds" %(usage/3600, upTime)) elif (clusterId == 0x00f6): clusterData(data['source_addr_long'],clusterId,data['rf_data']) print '' print "Identify Message" print "Sending init message" zb.send('tx_explicit', dest_addr_long = data['source_addr_long'], dest_addr = data['source_addr'], src_endpoint = '\x00', dest_endpoint = '\x02', cluster = '\x00\xf0', profile = '\xc2\x16', data = '\x19\x41\xfa\x00\x01' ) elif (clusterId == 0x00f3): clusterData(data['source_addr_long'],clusterId,data['rf_data']) print ' Key Fob Button', status = data['rf_data'] print ord(status[3]), if status[2] == '\x01': print 'Closed', elif status[2] == '\x00': print 'Open', else: print 'Unknown', print 'Counter', unpack('<H',status[5:7])[0], print '' pass elif (clusterId == 0x00f2): clusterData(data['source_addr_long'],clusterId,data['rf_data']) print 'Tamper Switch Changed State' pass elif (clusterId == 0x00f0): clusterData(data['source_addr_long'],clusterId,data['rf_data']) # If the cluster cmd byte is 'xfb', it's a status if data['rf_data'][2] == '\xfb': status = data['rf_data'] # just to make typing easier if status[3] == '\x1f': print " Door Sensor", print str(float(unpack("<h", status[8:10])[0])\ / 100.0 * 1.8 + 32) + "F", elif status[3] == '\x1c': # Never found anything useful in this print "Power Switch", elif status[3] == '\x1d': print " Key Fob", print str(float(unpack("<h", status[8:10])[0])\ / 100.0 * 1.8 + 32) + "F", unpack('<I',status[4:8])[0] print 'Counter', unpack('<I',status[4:8])[0], elif status[3] == '\x1e': # I haven't figured out what this is yet # it comes from a door switch and the temperature # field is always ff ff, it may be an error # indication. pass else: print " Don't know this device yet", print '' pass elif (clusterId == 0x0500): # This is the security cluster clusterData(data['source_addr_long'],clusterId,data['rf_data']) # When the switch first connects, it come up in a state that needs # initialization, this command seems to take care of that. # So, look at the value of the data and send the command. if data['rf_data'][3:7] == '\x15\x00\x39\x10': print "sending initialization" zb.send('tx_explicit', dest_addr_long = data['source_addr_long'], dest_addr = data['source_addr'], src_endpoint = '\x00', dest_endpoint = '\x00', cluster = '\x05\x00', profile = '\xc2\x16', data = '\x11\x80\x00\x00\x05' ) # The switch state is in byte [3] and is a bitfield # bit 0 is the magnetic reed switch state # bit 3 is the tamper switch state switchState = ord(data['rf_data'][3]) if switchState & 0x04: print 'Tamper Switch Closed', else: print 'Tamper Switch Open', if switchState & 0x01: print 'Reed Switch Opened', else: print 'Reed Switch Closed', print '' pass else: print ("Unimplemented Profile ID") elif(data['id'] == 'route_record_indicator'): print("Route Record Indicator") else: print("some other type of packet") print(data) except: print "I didn't expect this error:", sys.exc_info()[0] traceback.print_exc() def showDevices(): print "Known Devices ************" for key in devices: print "".join("%02x " % ord(b) for b in key)+':', print "".join("%02x " % ord(b) for b in devices[key][0]) print "**************************" def sendSwitch(whereLong, whereShort, srcEndpoint, destEndpoint, clusterId, profileId, clusterCmd, databytes): payload = '\x11\x00' + clusterCmd + databytes zb.send('tx_explicit', dest_addr_long = whereLong, dest_addr = whereShort, src_endpoint = srcEndpoint, dest_endpoint = destEndpoint, cluster = clusterId, profile = profileId, data = payload ) def tryCommand(): # Try out commands here print '********* Sending Test Command' switchLongAddr ='\x00\x0d\x6f\x00\x03\xc2\x71\xcc' if switchLongAddr in devices: switchShortAddr = devices[switchLongAddr][0] print ' Switch Status' sendSwitch(switchLongAddr, switchShortAddr, '\x00', '\x02', '\x00\xee', '\xc2\x16', '\x01', '\x01') else: print 'Waiting for short address' #------------ XBee Stuff ------------------------- # this is the /dev/serial/by-id device for the USB card that holds the XBee ZIGBEEPORT = "/dev/serial/by-id/usb-FTDI_FT232R_USB_UART_A901QL3F-if00-port0" ZIGBEEBAUD_RATE = 9600 # Open serial port for use by the XBee ser = serial.Serial(ZIGBEEPORT, ZIGBEEBAUD_RATE) # The XBee addresses I'm dealing with BROADCAST = '\x00\x00\x00\x00\x00\x00\xff\xff' UNKNOWN = '\xff\xfe' # This is the 'I don't know' 16 bit address #------------------------------------------------- logging.basicConfig() # Create XBee library API object, which spawns a new thread zb = ZigBee(ser, callback=messageReceived) # create a queue to put the messages into so they can # be handled in turn without one interrupting the next. messageQueue = Queue.Queue(0) # A dictionary to put devices into as they show up devices = {} scheditem = BackgroundScheduler() scheditem.add_job(showDevices, 'interval', seconds=60) #scheditem.add_job(tryCommand, 'interval', seconds=15) scheditem.start() print ("started") notYet = True; firstTime = True; while True: try: if (firstTime): # this is in case I need some initialization in the # future firstTime = False if messageQueue.qsize() > 0: #print "getting message" message = messageQueue.get() handleMessage(message) messageQueue.task_done(); time.sleep(0.1) #print ("tick") # This is here to let you know it's alive sys.stdout.flush() # if you're running non interactive, do this except IndexError: print "empty line" except NameError as e: print "NameError:", print e.message.split("'")[1] except KeyboardInterrupt: print ("Keyboard interrupt") break except: print ("I didn't expect this error:", sys.exc_info()[0]) traceback.print_exc() break print ("After the while loop") # halt() must be called before closing the serial # port in order to ensure proper thread shutdown scheditem.shutdown(wait=False) # shut down the apscheduler zb.halt() ser.close()
You'll notice that I keep a list of devices that have contacted the code. This is to allow you to tell what happened with various devices as the interaction continues. I log the long address, time and relevant details as they happen to help you understand how to deal with the incoming data. When you have several devices, the log can become a bit overpowering since nothing stops sending, it just goes on forever. Here's a bit of the logging so you can get an idea what it looks like:
started 1432598997 000d6f0004510782 clid 00f0-09 9e fb 1f bf 9e 7a 0a b7 0b c4 01 af f4 03 02 Door Sensor 85.982F 1432598998 000d6f0003cf0e5b clid 00ef-09 00 81 00 00 Current Usage: 0 1432598999 000d6f000283fa4e clid 00f0-09 a2 fb 1d 3c a2 e5 03 bc 0b 00 00 a7 fb 03 00 Key Fob 86.072F Counter 65380924 1432599000 000d6f00025886b3 clid 00ef-09 00 81 87 00 Current Usage: 135 1432599000 000d6f00025886b3 clid 00f0-09 00 fb 1c 22 dd 95 83 66 32 00 00 b5 b0 01 00 Power Switch 1432599000 000d6f000237b25a clid 00ef-09 00 81 01 00 Current Usage: 1 1432599003 000d6f000258a4cc clid 00ef-09 00 81 00 00 Current Usage: 0 1432599005 000d6f0002547a5d clid 00ef-09 00 81 00 00 Current Usage: 0 1432599007 000d6f0002547a5d clid 00f0-09 00 fb 1c 2b 09 b3 1a 66 32 00 00 b7 fe 01 00 Power Switch 1432599008 000d6f0003cf0e5b clid 00ef-09 00 81 00 00 Current Usage: 0 1432599009 000d6f00025886b3 clid 00ef-09 00 82 67 a4 a5 72 80 e5 20 01 00 Switch Minute Stats: Usage, 148 Watt Hours; Uptime, 18933120 Seconds 1432599010 000d6f00025886b3 clid 00ef-09 00 81 87 00 Current Usage: 135 1432599010 000d6f000237b25a clid 00ef-09 00 81 01 00 Current Usage: 1 1432599013 000d6f000258a4cc clid 00ef-09 00 82 f7 ea 21 00 b8 bc 03 00 00 Switch Minute Stats: Usage, 0 Watt Hours; Uptime, 244920 Seconds 1432599013 000d6f000258a4cc clid 00ef-09 00 81 00 00 Current Usage: 0 1432599015 000d6f0002547a5d clid 00ef-09 00 81 00 00 Current Usage: 0 1432599018 000d6f0003cf0e5b clid 00ef-09 00 81 00 00 Current Usage: 0 1432599020 000d6f00025886b3 clid 00ef-09 00 81 87 00 Current Usage: 135 1432599020 000d6f000237b25a clid 00ef-09 00 82 a9 3d 8a 1f 40 e9 20 01 00 Switch Minute Stats: Usage, 40 Watt Hours; Uptime, 18934080 Seconds 1432599020 000d6f0003cf0e5b clid 00f0-09 00 fb 1c 34 a9 6e 64 fb 31 00 00 ba b7 01 00 Power Switch 1432599020 000d6f000237b25a clid 00ef-09 00 81 01 00 Current Usage: 1 1432599023 000d6f000237b25a clid 00f0-09 00 fb 1c 25 0d a5 83 4a 32 00 00 b4 b4 01 00 Power Switch 1432599023 000d6f000258a4cc clid 00ef-09 00 81 00 00 Current Usage: 0 1432599025 000d6f0002547a5d clid 00ef-09 91 82 40 35 ff 00 d4 ac 06 00 00 Switch Minute Stats: Usage, 1 Watt Hours; Uptime, 437460 Seconds 1432599025 000d6f0002547a5d clid 00ef-09 00 81 00 00 Current Usage: 0 1432599025 000d6f000258a4cc clid 00f0-09 00 fb 1c 18 11 f3 0e 66 32 00 00 bd b3 01 00 Power Switch 1432599028 000d6f0003cf0e5b clid 00ef-09 00 81 00 00 Current Usage: 0 1432599030 000d6f00025886b3 clid 00ef-09 00 81 87 00 Current Usage: 135 1432599030 000d6f00025886b3 clid 00f0-09 00 fb 1c 22 55 96 83 66 32 00 00 b6 b6 01 00 Power Switch 1432599030 000d6f000237b25a clid 00ef-09 00 81 01 00 Current Usage: 1 1432599033 000d6f000258a4cc clid 00ef-09 00 81 00 00 Current Usage: 0 1432599035 000d6f0002547a5d clid 00ef-09 00 81 00 00 Current Usage: 0 1432599036 000d6f0002547a5d clid 00f0-09 00 fb 1c 2b 81 b3 1a 66 32 00 00 b7 fd 01 00 Power Switch 1432599038 000d6f0003cf0e5b clid 00ef-09 00 82 38 42 54 00 bc 1b 19 00 00 Switch Minute Stats: Usage, 0 Watt Hours; Uptime, 1645500 Seconds 1432599038 000d6f0003cf0e5b clid 00ef-09 00 81 00 00 Current Usage: 0 1432599040 000d6f00025886b3 clid 00ef-09 00 81 87 00 Current Usage: 135 1432599040 000d6f000237b25a clid 00ef-09 00 81 01 00 Current Usage: 1 1432599043 000d6f000258a4cc clid 00ef-09 00 81 00 00 Current Usage: 0 1432599045 000d6f0002547a5d clid 00ef-09 00 81 00 00 Current Usage: 0 1432599048 000d6f0003cf0e5b clid 00ef-09 00 81 00 00 Current Usage: 0 1432599050 000d6f00025886b3 clid 00ef-09 00 81 87 00 Current Usage: 135 1432599050 000d6f0003cf0e5b clid 00f0-09 00 fb 1c 34 21 6f 64 fb 31 00 00 ba bb 01 00 Power Switch 1432599050 000d6f000237b25a clid 00ef-09 00 81 01 00 Current Usage: 1 1432599053 000d6f000237b25a clid 00f0-09 00 fb 1c 25 85 a5 83 4a 32 00 00 b4 a4 01 00 Power Switch 1432599054 000d6f000258a4cc clid 00ef-09 00 81 00 00 Current Usage: 0 1432599055 000d6f0002547a5d clid 00ef-09 00 81 00 00 Current Usage: 0 1432599055 000d6f000258a4cc clid 00f0-09 00 fb 1c 18 89 f3 0e 66 32 00 00 bd b1 01 00 Power Switch Known Devices ************ 00 0d 6f 00 02 83 fa 4e : 92 26 00 0d 6f 00 04 51 07 82 : a9 3b 00 0d 6f 00 02 58 a4 cc : e1 3c 00 0d 6f 00 02 58 86 b3 : 27 f6 00 0d 6f 00 02 37 b2 5a : 2b d1 00 0d 6f 00 02 54 7a 5d : 41 c7 00 0d 6f 00 03 cf 0e 5b : 79 c5 ************************** 1432599058 000d6f0003cf0e5b clid 00ef-09 00 81 00 00 Current Usage: 0 1432599060 000d6f00025886b3 clid 00ef-09 00 81 87 00 Current Usage: 135 1432599060 000d6f00025886b3 clid 00f0-09 00 fb 1c 22 cd 96 83 66 32 00 00 b6 c1 01 00 Power Switch
Remember, this runs on a Raspberry Pi, I haven't tried it on anything else. I have my XBee plugged into a USB port on the Pi, if you're using the serial port, just change the port in the code, that should be all that is needed. If you don't know which usb port to use, there's a ton of articles on the web that will help you find the port, just look around a bit, or to see how I did it look here <link>. What I'm hoping for is a couple of folk that want to use these nicely priced and well made devices with their own code for some project to grab it and give me some feedback on the interaction. It would be nice to see someone else running this code.
Have fun.
The next part to this project is here <link>
Subscribe to:
Posts (Atom)