So far, I've been a fan of cloud services. I say, "so far," because it's a great idea on the surface. For free, or a small fee, web businesses provide storage, blogging, graphs, and many specialized services. It seems like the perfect solution to problems that we little folk have. Some of us want to investigate power (me), others want to share pictures, still others want long winded political diatribes, and there are sites out there that cater to us.
But (remember, there's always a 'but'), the darn providers want to constantly change what they provide and charge ever increasing amounts of money for the services. I've talked about my disappointment with Xively (Cosm, Pachube) over the years as they got bought out by various companies and abandoned the ideals and price point they started with. Some of the services start off with a bang and dwindle to a whimper decreasing services down to an almost unusable level. Others unexpectedly change things and don't let you know in advance, or even after the fact ... you have to find out because things quit working (usually over the weekend), and it takes time and patience to ferret out what the heck they did.
In the last couple of weeks I've encountered this annoying activity in regards to my charts that are displayed on this site. One day (a Monday naturally), all my charts based on Xively storage stopped working. It took me a while to figure out what happened and fix it. It seems that Xively stopped responding to the old Pachube URLs and didn't give any warning at all. I checked their blog, site, twitter account and my spam folder, nothing. So, I had to visit each use of the charts and change the word 'pachube' to 'xively' to make it work. Not a bad thing, and it wouldn't have mattered if they dropped me a note or put something somewhere that this was going to happen. But, it was really annoying to have it sneak up like that.
Then, the charts stopped working again. Nope, it wasn't Xively messing around, this time it was dropbox. They tightened up the security and forced their old http links into https. This is normally considered a good thing, but when you have a number of embedded charts that all look at http, it is a pain to change all of them. Especially when they didn't say a word about it. Nope, it didn't come in the email, no mention anywhere I could find.
After chasing down the various references, I got them fixed and working again (I think).
This has happened to me before; actually several times, google dropped support for their .png charts, I had to scramble to find another way. A cloud storage service disappeared, that's how I decided to move to dropbox. One of the quicken services became Mint and the transition sucked for me; I just dropped that idea altogether. Etc.
I still think Cloud Services are a good idea, but the specific implementations seem to be ridden with various problems we have to adapt to. The biggest being some corporate know-it-all that decides to change things without communicating it. True, it is their service and they can do what they want. I have the choice of staying with them or leaving for something else that fits my needs or doesn't tick me off as much.
However, I also have the choice of grabbing one of the great new little computers, hanging a big ol' drive on it that I can pick up for a song these days, and rolling my own in-home cloud services. I mean, how hard can it be to record the data I send up the wire locally? A nice free database manager (there are dozens), a little front end code that listens to http, and away I go. Sure, it'll expose a machine to the internet if I want to make it public (I do) and I'd have to actually invest in a real router to isolate the rest of the house, but I wouldn't be subject to the changing whims of some corporate MBA that will only hold the job for a few months before leaving for greener pastures.
I haven't decided to abandon the cloud yet, but it's getting more and more compelling all the time.
Monday, June 30, 2014
Friday, June 20, 2014
I'll try to explain XBee API Mode and How to Set it Up
I've gotten hundreds of questions on XBee API mode. API mode is necessary for setting up an XBee network of more than three devices because AT mode (also called transparent) will cause fragmentation and mixing of data over time. I have a post somewhere about this that explains my experience when I finally exceeded what the XBee was capable of when using AT mode <link>.
So, what is API mode? It's simply a protocol between the serial port on the XBee and your controller. Be it Raspberry Pi, Arduino, Galileo, whatever, you talk to an XBee through a serial port and what goes over the serial port isn't what the XBees send amongst themselves. First, a simple picture:
I'm not an artist, so use your imagination. The term 'API mode' refers only to the two serial links that connect your controllers to the XBee. What goes over the RF link is not affected and shouldn't be; that's something that Digi takes care of quite nicely.
In AT mode, what you send out the port is what you want to be sent to the other controller. You want, "Hello World," that's exactly what you send. The XBee will collect the characters and put them in an RF packet and send them to the other XBee. If the other XBee is in AT mode as well, it will parse the characters out and send them through the serial port to the other controller. Simple, easy to understand, and the way almost everyone starts out with the XBees.
Now a little bit about API mode. API mode allows you to shape a packet complete with checksum, destination address, and other things that give you control over what goes on. It also allows you confidence that what you send gets to the other end complete and in one piece. See, AT mode will send some of it, wait, then send the rest of it. This depends on traffic and RF noise, so it's possible to get part of a message, then the rest of it later. If you have a number of XBees, you'll actually get messages that have been mixed together and made unusable. You get all the stuff that was sent, but it may be jumbled up. API mode is the way to go after you've played with AT mode some to get a feel for things.
So, how to you set up an XBee to transmit in API mode? I only work with Series 2 and later XBees, so everything here applies to them, if you're working with series 1, you're going to have to look somewhere else for examples. Remember that API mode only applies to the serial links; I've labeled them in the drawing, so there are two things you have to change to get API mode on the XBee: the software you load on the device, and the serial interface you choose.
I use modem setting 'XB24-ZB' and Function Set 'ZIGBEE COORDINATOR API'. These are found in the Modem Configuration tab of XCTU up at the top. I always choose that latest software version on the far right. Here's a picture to help you find it:
This software, once you load it on the XBee will give you API mode. The XBees come defaulted to AT mode, so you'll have to download XCTU and change them yourself. Of course, if you're programming a router, choose 'ZIGBEE ROUTER API' as the 'Function Set.'
Now you have to change the various parameters to be specific for your usage. Things like Pan ID, Channel, that kind of stuff. I address these things in other posts, so look them up there, or use one of the jillion tutorials out there. Once you get that stuff in, you can set which API mode you want to use. On the Arduino using Andrew Rapp's XBee library, you'll need to use API mode 2. For the Raspberry Pi using the XBee package, you'll need to use API mode 1. Don't worry, they will work fine together. Remember above where I told you API mode only affects the serial link to the controller and not the RF link between XBees? This is the beauty of these little devices, the mode doesn't change how the XBees talk to each other, just how they talk to the controller.
API mode is set under 'Serial Interfacing' a little bit down the screen, so scroll down until you find it. The values are (duh) 0, 1, or 2. Zero means no API, 1 is for API mode 1, and 2 is for API mode 2. What's the difference? I talk about this and the interaction of the various modes here <link>. Why do I need different API modes on the different controllers? Because the authors of the libraries wrote them that way and we have to adjust ... or heaven forbid ... write our own code to handle it.
Write the parameters to the XBee and you have now set up the XBee to work in API mode using whichever mode ( 1 or 2 ) is appropriate. In case you need to save this configuration, there is a profile save button on the screen. Use it to keep the various profiles you try over time. You can simply reload it and avoid all of this next time.
One other thing: now you have to change the settings on the first screen of XCTU to talk to it properly.
About half way down the screen there's a box 'API,' check the 'Enable API' setting, and depending on how you configured the XBee, check the 'Use escape characters' box. In API mode 1, you don't check it; in API mode 2, you do.
Now you have to change the code you're running on the controllers to use the new mode. It's worth it because you can check the checksum on a message to see if it got clobbered on its way to you, address specific XBees so you don't have unnecessary traffic. Use end devices that don't have a controller attached, just use the native pins on the XBee. It makes the little devices much more versatile. For examples, I have posts about some of the libraries on this blog, Just type XBee in the search box on the upper left and look around.
So, what is API mode? It's simply a protocol between the serial port on the XBee and your controller. Be it Raspberry Pi, Arduino, Galileo, whatever, you talk to an XBee through a serial port and what goes over the serial port isn't what the XBees send amongst themselves. First, a simple picture:
In AT mode, what you send out the port is what you want to be sent to the other controller. You want, "Hello World," that's exactly what you send. The XBee will collect the characters and put them in an RF packet and send them to the other XBee. If the other XBee is in AT mode as well, it will parse the characters out and send them through the serial port to the other controller. Simple, easy to understand, and the way almost everyone starts out with the XBees.
Now a little bit about API mode. API mode allows you to shape a packet complete with checksum, destination address, and other things that give you control over what goes on. It also allows you confidence that what you send gets to the other end complete and in one piece. See, AT mode will send some of it, wait, then send the rest of it. This depends on traffic and RF noise, so it's possible to get part of a message, then the rest of it later. If you have a number of XBees, you'll actually get messages that have been mixed together and made unusable. You get all the stuff that was sent, but it may be jumbled up. API mode is the way to go after you've played with AT mode some to get a feel for things.
So, how to you set up an XBee to transmit in API mode? I only work with Series 2 and later XBees, so everything here applies to them, if you're working with series 1, you're going to have to look somewhere else for examples. Remember that API mode only applies to the serial links; I've labeled them in the drawing, so there are two things you have to change to get API mode on the XBee: the software you load on the device, and the serial interface you choose.
I use modem setting 'XB24-ZB' and Function Set 'ZIGBEE COORDINATOR API'. These are found in the Modem Configuration tab of XCTU up at the top. I always choose that latest software version on the far right. Here's a picture to help you find it:
This software, once you load it on the XBee will give you API mode. The XBees come defaulted to AT mode, so you'll have to download XCTU and change them yourself. Of course, if you're programming a router, choose 'ZIGBEE ROUTER API' as the 'Function Set.'
Now you have to change the various parameters to be specific for your usage. Things like Pan ID, Channel, that kind of stuff. I address these things in other posts, so look them up there, or use one of the jillion tutorials out there. Once you get that stuff in, you can set which API mode you want to use. On the Arduino using Andrew Rapp's XBee library, you'll need to use API mode 2. For the Raspberry Pi using the XBee package, you'll need to use API mode 1. Don't worry, they will work fine together. Remember above where I told you API mode only affects the serial link to the controller and not the RF link between XBees? This is the beauty of these little devices, the mode doesn't change how the XBees talk to each other, just how they talk to the controller.
API mode is set under 'Serial Interfacing' a little bit down the screen, so scroll down until you find it. The values are (duh) 0, 1, or 2. Zero means no API, 1 is for API mode 1, and 2 is for API mode 2. What's the difference? I talk about this and the interaction of the various modes here <link>. Why do I need different API modes on the different controllers? Because the authors of the libraries wrote them that way and we have to adjust ... or heaven forbid ... write our own code to handle it.
Write the parameters to the XBee and you have now set up the XBee to work in API mode using whichever mode ( 1 or 2 ) is appropriate. In case you need to save this configuration, there is a profile save button on the screen. Use it to keep the various profiles you try over time. You can simply reload it and avoid all of this next time.
One other thing: now you have to change the settings on the first screen of XCTU to talk to it properly.
About half way down the screen there's a box 'API,' check the 'Enable API' setting, and depending on how you configured the XBee, check the 'Use escape characters' box. In API mode 1, you don't check it; in API mode 2, you do.
Now you have to change the code you're running on the controllers to use the new mode. It's worth it because you can check the checksum on a message to see if it got clobbered on its way to you, address specific XBees so you don't have unnecessary traffic. Use end devices that don't have a controller attached, just use the native pins on the XBee. It makes the little devices much more versatile. For examples, I have posts about some of the libraries on this blog, Just type XBee in the search box on the upper left and look around.
Saturday, June 14, 2014
Cell Phone Charging and Power Usage.
One of the comments on this blog made me start wondering about the little parasitic devices we have all over the house. I've always assumed that they drew so little power they wouldn't matter when compared to to the kilowatt guzzling motors and heating elements we have in the larger appliances. One little device that annoys me is the cell phone charger. Every time the phone gets to a full charge it tells me to unplug the charger to save energy. Sheesh, leave me alone, let me worry about how much power I'm using.
However, the commenter said his measuring device recorded 15 watts for several of his devices. If I can confirm that, I may have to think about doing something since I have a bunch of them around the house. So, I'll take on an annoyance and see what the power usage really is for my cell phone charger.
However, the commenter said his measuring device recorded 15 watts for several of his devices. If I can confirm that, I may have to think about doing something since I have a bunch of them around the house. So, I'll take on an annoyance and see what the power usage really is for my cell phone charger.
This little phone sucks 12 watts when it first starts out, so I need a really good wall wart that can supply over two amps to get the quickest charge. Since I used the 'genuine' Samsung charger for this test, I let the phone drain down to (approximately) the same point and tried one of those 'Samsung' chargers that are available on Ebay.
Don't let the scale confuse you, this charger never gets over 1 watt. Since my granularity (using this monitor device) is 1 watt, it could have gone a little higher or lower and still read this way, so I plugged in an amp meter in series and watched a while and it didn't ever go over 500 mA. It doesn't have the stair steps of the real Samsung charger and took much longer ( less than an hour compared to over 2.5). It looks like buying a real brand name may result in much, much better performance. However, how the heck do you tell if it's really from the manufacturer? Both of them are labeled 'Samsung' and they are the same form factor. The ratings on the side are the same, so how do us folk out here in the world tell the difference?
I don't have an answer to this, and I'm certainly not going to buy a few hundred different ones and try them out. I'm seriously thinking about making a load to test these things before I try to depend on them for anything. If they fail the test, I'll do some serious complaining to the supplier. The other thing I'm thinking about is putting together a power supply that will give me 5V at three amps reliably. This would be useful to see how the charge characteristics of the phone look when it has enough current available.
Just for fun, I weighed each of them. The real Samsung charger weighed 37 grams and the other one came in at 25 grams. The real one has 12 grams more stuff in it, or thicker plastic. Since I plan on using the fake for other things, I'm not going to dismantle it ... yet. But here's a picture of the two of them:
The fake is the black one.
Just to let you folk know though, there are two differences between them. 1. The real one has a UL certification on it, the fake doesn't. That's really easy to overcome, they simply add another certification stamp to it. 2. The fake says 5.0 volt at 2 amp, and the real one says 5.3 volt at 2 amp. Once again, that's easy to overcome. I've already mentioned the weight, but no supplier tells you the weight of the device.
Anyway, I started this as an investigation into parasitic power and wound up researching chargers and their capabilities. Sigh. At any rate, let the buyer beware.
Saturday, June 7, 2014
Taking apart the Iris Smart Switch
I decided to take the plunge and finish off the 120V appliances I want to monitor right now. There will probably be others in the future, but for now the two freezers and refrigerator will be the devices I keep an eye on. I also want another Iris Smart Switch to wander around the house with. It was nice to be able to evaluate the new smoker <link>, and I'm sure I'll want to do that again with some other appliance (TV, wall warts, maybe a Raspberry Pi) from time to time.
This time though, I'll finish the software to monitor the appliances and take the switch apart to modify it to make sure the appliances can't be turned off accidentally. It's been a concern of mine that the switch can be both locally and remotely controlled and my major appliances are fed through it. I don't want a power surge or EMF pulse to shut down my freezers. So, I got out the screwdriver and a knife. I peeled the label covering one of the screws and took the thing apart.
I was expecting something like the X10 devices I've dismantled over the years, and was I ever surprised, this thing is really well made:
The relay is a single pole, double throw and they use both sets of contacts. I was impressed by the construction and may, someday, take the board out and use it for control of something else. For now, having it at the plug is really great.
After I shorted across the relay pins --- go look up the relay to get the pins <link>, I don't want to be responsible for you messing it up and screwing up the switch --- I put it back together and started working on the code. The way I did it is, if a new device shows up, I put an entry in my Sqlite3 database holding the fields I'm interested in (all I use) and assigned it the name of 'unknown'. Then I go ahead and let the switch join. It will immediately start sending data, and I save it to the new record in the database. Then using the command line interface for Sqlite3, I just give it a name and it is now part of my network. If you want to store it in some other fashion, just change the code to accommodate whatever technique you are using. However, I really like the Sqlite3 database, it serves as storage for all my devices and has worked like a champ for months now. If you're doing this on an Arduino, you can use the non-volatile storage capability. I'm not going to port this code to the Arduino though, I have no need for it there.
Sure, the chart looks too busy to understand, but remember you can click on the legend items on the bottom and turn off whatever you don't want to look at right now. It looks like my garage freezer is roughly equivalent to a 50W light bulb running all the time. I guess I need to figure out what that costs me, but it takes a spreadsheet to do it. I have seasonal rates and demand billing, that will take some thought.
So, I'll add one more device to float around the house measuring things that may catch my interest from time to time. The kill-a-watt is cool, but you don't get a feel for how a particular device will behave over time. The kill-a-watt didn't tell me that the freezer defrost timer was hitting during the peak period when it could just as well operate during off-peak periods.
Now I have to open the other two devices and install a jumper, have fun.
This time though, I'll finish the software to monitor the appliances and take the switch apart to modify it to make sure the appliances can't be turned off accidentally. It's been a concern of mine that the switch can be both locally and remotely controlled and my major appliances are fed through it. I don't want a power surge or EMF pulse to shut down my freezers. So, I got out the screwdriver and a knife. I peeled the label covering one of the screws and took the thing apart.
I was expecting something like the X10 devices I've dismantled over the years, and was I ever surprised, this thing is really well made:
Notice that there's a well laid out circuit board, nice large wires to handle the load, and substantial connectors for the power in and out. Right off I noticed that they had a resistor in the circuit for power. Yes, they're using a real shunt resistor to measure power. Talk about old school, tried and true, easy to work with design. There's a pretty substantial set of filtering caps to keep noise out of the circuitry and a 16A Omron relay for control. The point I wanted was where the power is broken by the relay; I just need to short across the relay connections and I'll be done.
For the folk out there that want to see the rest of it, here's the back side of the board:
After I shorted across the relay pins --- go look up the relay to get the pins <link>, I don't want to be responsible for you messing it up and screwing up the switch --- I put it back together and started working on the code. The way I did it is, if a new device shows up, I put an entry in my Sqlite3 database holding the fields I'm interested in (all I use) and assigned it the name of 'unknown'. Then I go ahead and let the switch join. It will immediately start sending data, and I save it to the new record in the database. Then using the command line interface for Sqlite3, I just give it a name and it is now part of my network. If you want to store it in some other fashion, just change the code to accommodate whatever technique you are using. However, I really like the Sqlite3 database, it serves as storage for all my devices and has worked like a champ for months now. If you're doing this on an Arduino, you can use the non-volatile storage capability. I'm not going to port this code to the Arduino though, I have no need for it there.
The Python Script
#! /usr/bin/python # This is the an implementation of monitoring the Lowe's Iris Smart # Switch that I use. It will join with a switch and does NOT allow you # to control the switch # # This version has been adapted to support more than one switch and will # add a new record to my database to hold the data. Adapt it as you need # to. # # Have fun from xbee import ZigBee from apscheduler.scheduler import Scheduler import logging import datetime import time import serial import sys import shlex import sqlite3 #------------------------------------------------- # the database where I'm storing stuff DATABASE='/home/pi/database/desert-home' # on the Raspberry Pi the serial port is ttyAMA0 XBEEPORT = '/dev/ttyUSB1' XBEEBAUD_RATE = 9600 # The XBee addresses I'm dealing with BROADCAST = '\x00\x00\x00\x00\x00\x00\xff\xff' UNKNOWN = '\xff\xfe' # This is the 'I don't know' 16 bit address #------------------------------------------------- logging.basicConfig() # this is the only way I could think of to get the address strings to store. # I take the ord() to get a number, convert to hex, then take the 3 to end # characters and pad them with zero and finally put the '0x' back on the front # I put spaces in between each hex character to make it easier to read. This # left an extra space at the end, so I slice it off in the return statement. # I hope this makes it easier to grab it out of the database when needed def addrToString(funnyAddrString): hexified = '' for i in funnyAddrString: hexified += '0x' + hex(ord(i))[2:].zfill(2) + ' ' return hexified[:-1] #------------ XBee Stuff ------------------------- # Open serial port for use by the XBee ser = serial.Serial(XBEEPORT, XBEEBAUD_RATE) # this is a call back function. When a message # comes in this function will get the data def messageReceived(data): #print 'gotta packet' #print data clusterId = (ord(data['cluster'][0])*256) + ord(data['cluster'][1]) #print 'Cluster ID:', hex(clusterId), if (clusterId == 0x13): # This is the device announce message. # due to timing problems with the switch itself, I don't # respond to this message, I save the response for later after the # Match Descriptor request comes in. You'll see it down below. # if you want to see the data that came in with this message, just # uncomment the 'print data' comment up above print 'Device Announce Message' elif (clusterId == 0x8005): # this is the Active Endpoint Response This message tells you # what the device can do, but it isn't constructed correctly to match # what the switch can do according to the spec. This is another # message that gets it's response after I receive the Match Descriptor print 'Active Endpoint Response' elif (clusterId == 0x0006): # Match Descriptor Request; this is the point where I finally # respond to the switch. Several messages are sent to cause the # switch to join with the controller at a network level and to cause # it to regard this controller as valid. # # First the Active Endpoint Request payload1 = '\x00\x00' zb.send('tx_explicit', dest_addr_long = data['source_addr_long'], dest_addr = data['source_addr'], src_endpoint = '\x00', dest_endpoint = '\x00', cluster = '\x00\x05', profile = '\x00\x00', data = payload1 ) print 'sent Active Endpoint' # Now the Match Descriptor Response payload2 = '\x00\x00\x00\x00\x01\x02' zb.send('tx_explicit', dest_addr_long = data['source_addr_long'], dest_addr = data['source_addr'], src_endpoint = '\x00', dest_endpoint = '\x00', cluster = '\x80\x06', profile = '\x00\x00', data = payload2 ) print 'Sent Match Descriptor' # Now there are two messages directed at the hardware # code (rather than the network code. The switch has to # receive both of these to stay joined. payload3 = '\x11\x01\x01' zb.send('tx_explicit', dest_addr_long = data['source_addr_long'], dest_addr = data['source_addr'], src_endpoint = '\x00', dest_endpoint = '\x02', cluster = '\x00\xf6', profile = '\xc2\x16', data = payload2 ) payload4 = '\x19\x01\xfa\x00\x01' zb.send('tx_explicit', dest_addr_long = data['source_addr_long'], dest_addr = data['source_addr'], src_endpoint = '\x00', dest_endpoint = '\x02', cluster = '\x00\xf0', profile = '\xc2\x16', data = payload4 ) print 'Sent hardware join messages' # now that it should have joined, I'll add a record to the database to # hold the status. I'll just name the device 'unknown' so it can # be updated by hand using sqlite3 directly. If the device already exists, # I'll leave the name alone and just use the existing record # Yes, this means you'll have to go into the database and assign it a name # dbconn = sqlite3.connect(DATABASE) c = dbconn.cursor() # See if the device is already in the database c.execute("select name from smartswitch " "where longaddress = ?; ", (addrToString(data['source_addr_long']),)) switchrecord = c.fetchone() if switchrecord is not None: print "Device %s is rejoining the network" %(switchrecord[0]) else: print "Adding new device" c.execute("insert into smartswitch(name,longaddress, shortaddress, status, watts, twatts, utime)" "values (?, ?, ?, ?, ?, ?, ?);", ('unknown', addrToString(data['source_addr_long']), addrToString(data['source_addr']), 'unknown', '0', '0', time.strftime("%A, %B, %d at %H:%M:%S"))) dbconn.commit() dbconn.close() elif (clusterId == 0xef): clusterCmd = ord(data['rf_data'][2]) if (clusterCmd == 0x81): usage = ord(data['rf_data'][3]) + (ord(data['rf_data'][4]) * 256) dbconn = sqlite3.connect(DATABASE) c = dbconn.cursor() # This is commented out because I don't need the name # unless I'm debugging. # get device name from database #c.execute("select name from smartswitch " # "where longaddress = ?; ", # (addrToString(data['source_addr_long']),)) #name = c.fetchone()[0].capitalize() #print "%s Instaneous Power, %d Watts" %(name, usage) # do database updates c.execute("update smartswitch " "set watts = ?, " "shortaddress = ?, " "utime = ? where longaddress = ?; ", (usage, addrToString(data['source_addr']), time.strftime("%A, %B, %d at %H:%M:%S"), addrToString(data['source_addr_long']))) dbconn.commit() dbconn.close() elif (clusterCmd == 0x82): usage = (ord(data['rf_data'][3]) + (ord(data['rf_data'][4]) * 256) + (ord(data['rf_data'][5]) * 256 * 256) + (ord(data['rf_data'][6]) * 256 * 256 * 256) ) upTime = (ord(data['rf_data'][7]) + (ord(data['rf_data'][8]) * 256) + (ord(data['rf_data'][9]) * 256 * 256) + (ord(data['rf_data'][10]) * 256 * 256 * 256) ) dbconn = sqlite3.connect(DATABASE) c = dbconn.cursor() c.execute("select name from smartswitch " "where longaddress = ?; ", (addrToString(data['source_addr_long']),)) name = c.fetchone()[0].capitalize() print "%s Minute Stats: Usage, %d Watt Hours; Uptime, %d Seconds" %(name, usage/3600, upTime) # update database stuff c.execute("update smartswitch " "set twatts = ?, " "shortaddress = ?, " "utime = ? where longaddress = ?; ", (usage, addrToString(data['source_addr']), time.strftime("%A, %B, %d at %H:%M:%S"), addrToString(data['source_addr_long']))) dbconn.commit() dbconn.close() elif (clusterId == 0xf0): clusterCmd = ord(data['rf_data'][2]) # print "Cluster Cmd:", hex(clusterCmd), # if (clusterCmd == 0xfb): #print "Temperature ??" # else: #print "Unimplemented" elif (clusterId == 0xf6): clusterCmd = ord(data['rf_data'][2]) # if (clusterCmd == 0xfd): # pass #print "RSSI value:", ord(data['rf_data'][3]) # elif (clusterCmd == 0xfe): # pass #print "Version Information" # else: # pass #print data['rf_data'] elif (clusterId == 0xee): clusterCmd = ord(data['rf_data'][2]) status = '' if (clusterCmd == 0x80): if (ord(data['rf_data'][3]) & 0x01): status = "ON" else: status = "OFF" dbconn = sqlite3.connect(DATABASE) c = dbconn.cursor() c.execute("select name from smartswitch " "where longaddress = ?; ", (addrToString(data['source_addr_long']),)) print c.fetchone()[0].capitalize(), print "Switch is", status c.execute("update smartswitch " "set status = ?, " "shortaddress = ?, " "utime = ? where longaddress = ?; ", (status, addrToString(data['source_addr']), time.strftime("%A, %B, %d at %H:%M:%S"), addrToString(data['source_addr_long']))) dbconn.commit() dbconn.close() else: print "Unimplemented Cluster ID", hex(clusterId) print def sendSwitch(whereLong, whereShort, srcEndpoint, destEndpoint, clusterId, profileId, clusterCmd, databytes): payload = '\x11\x00' + clusterCmd + databytes # print 'payload', # for c in payload: # print hex(ord(c)), # print # print 'long address:', # for c in whereLong: # print hex(ord(c)), # print zb.send('tx_explicit', dest_addr_long = whereLong, dest_addr = whereShort, src_endpoint = srcEndpoint, dest_endpoint = destEndpoint, cluster = clusterId, profile = profileId, data = payload ) # This just puts a time stamp in the log file for tracking def timeInLog(): print time.strftime("%A, %B, %d at %H:%M:%S") #------------------If you want to schedule something to happen ----- scheditem = Scheduler() scheditem.start() scheditem.add_interval_job(timeInLog, minutes=15) #----------------------------------------------------------------- # Create XBee library API object, which spawns a new thread zb = ZigBee(ser, callback=messageReceived) print "started at ", time.strftime("%A, %B, %d at %H:%M:%S") while True: try: time.sleep(0.1) sys.stdout.flush() # if you're running non interactive, do this except KeyboardInterrupt: print "Keyboard interrupt" break except: print "Unexpected error:", sys.exc_info()[0] break print "After the while loop" # halt() must be called before closing the serial # port in order to ensure proper thread shutdown zb.halt() ser.close()
Of course, I have to update my storage out there to hold the new appliance and start charting it. That means, for now, updating my legacy feeds on Xively and then adding the new feed to my appliance chart. Then when I get (yet) another one of these, I'll have to make similar updates for it. I know, I could spend a few days coming up with a way to automate this, but why bother? It only takes an hour or so to add a new monitor device and I don't have any current plans for a new one. Here's the latest chart, notice I can now monitor the freezer out in the garage.
Sure, the chart looks too busy to understand, but remember you can click on the legend items on the bottom and turn off whatever you don't want to look at right now. It looks like my garage freezer is roughly equivalent to a 50W light bulb running all the time. I guess I need to figure out what that costs me, but it takes a spreadsheet to do it. I have seasonal rates and demand billing, that will take some thought.
So, I'll add one more device to float around the house measuring things that may catch my interest from time to time. The kill-a-watt is cool, but you don't get a feel for how a particular device will behave over time. The kill-a-watt didn't tell me that the freezer defrost timer was hitting during the peak period when it could just as well operate during off-peak periods.
Now I have to open the other two devices and install a jumper, have fun.
Friday, June 6, 2014
I Got My Smart Meter - North of Phoenix, AZ
Yep, today I got my smart meter. I called the power company yesterday and asked them the questions I had and guess what ... they didn't have the answers. Are you surprised (not!)? No, I'm not a health nut; I realize that the frequency of the transmissions is non ionizing, and that the intensity decreases with the square of the distance, so this tiny radio on the side of my garage is about 1/1000th as dangerous as a wayward Lego on the floor.
No, what I wanted to know is:
Is it Zigbee? No, they chose another transmission and protocol method. I don't know what, because the tech didn't either, however it is a store forward mesh network that can use other nearby meters to forward the data to a central collector.
What frequency range does it use: It's in the 900mHz band.
How often does it report? The tech was unclear on that, but the total time is only seconds a day which leads me to believe that it's on an hourly basis.
Who made it? The meter is the common Elster REX2. Which negated a whole series of questions about what technique was used to measure the power coming into the house. I'll talk more about this a little further down.
Were they any good? He thought they were accurate and seemed to be holding up well in the heat, but the cover tended to glaze over from the sun and there seemed to be a higher replacement rate for not reporting than the old meter. This is a little strange though since the old meter didn't report at all; they had to come out and actually read it. I suspect he was talking about a seemingly unusual rate of not reporting for the meters he had installed.
Where was the collector for the meters? It's down the road and around the corner roughly a mile from my house. The meters are rated for six miles on their own, so the data may not have to be relayed at all if the range claims are true.
How does the data get to APS (the power company)? At the collector it is forwarded to Verizon and uses their facilities to get it to the power company data center. Apparently, they use both cellular and wired depending on where the collector is and what facilities are available. Out where I am, they use cellular links to forward the data.
Did it have a pulse output? NO DARN IT. I was looking forward to setting up hardware to read the pulses to see how accurate my measurements are over the long haul. This was disappointing, but since I already measure the power myself, I don't have much to complain about.
So, I didn't gain much with the new meter. However the power company will collect data by usage time and I can prowl around their data and compare it to mine. If they show a significant difference, I might have to talk to them about it. Although, if they record less than I read, they won't be hearing from me.
What's fun though is that people have taken these meters apart and looked inside <link>. This article is absolutely great because it matches my meter exactly. I stole a couple of pictures from the article because I can't be sure it won't disappear at some point and I want to point out a couple of things:
This is the bottom plate of the meter. Notice the large wires to connect the supply to my house, they have a current transformer wrapped around them. This is almost exactly the way I measure power with my power monitor <link> The big difference is that I had to use two CTs to measure the power since the wires were too far apart to get one CT to go around them both. Nice to have some validation after so long. I've had to explain this a thousand times to various folk. Here's a picture of the CT with the wires after the author removed them from the housing:
Clever weren't they? The CT leads into a circuit board that measures the current and voltage to calculate the power usage. Darn, this is really starting to sound familiar.
There's a ton of conspiracy folk out there that see smart meters as a threat. They could become one at some point given an unrestricted power company and shoddy government oversight, but I'm not worried about that. I'm excited about the technology. This little guy can measure my power and send it up to APS (my power company) once it's there, I can grab it and do comparisons against my own measurements.
Heck, I'm using my power company as another cloud storage.
No, what I wanted to know is:
Is it Zigbee? No, they chose another transmission and protocol method. I don't know what, because the tech didn't either, however it is a store forward mesh network that can use other nearby meters to forward the data to a central collector.
What frequency range does it use: It's in the 900mHz band.
How often does it report? The tech was unclear on that, but the total time is only seconds a day which leads me to believe that it's on an hourly basis.
Who made it? The meter is the common Elster REX2. Which negated a whole series of questions about what technique was used to measure the power coming into the house. I'll talk more about this a little further down.
Were they any good? He thought they were accurate and seemed to be holding up well in the heat, but the cover tended to glaze over from the sun and there seemed to be a higher replacement rate for not reporting than the old meter. This is a little strange though since the old meter didn't report at all; they had to come out and actually read it. I suspect he was talking about a seemingly unusual rate of not reporting for the meters he had installed.
Where was the collector for the meters? It's down the road and around the corner roughly a mile from my house. The meters are rated for six miles on their own, so the data may not have to be relayed at all if the range claims are true.
How does the data get to APS (the power company)? At the collector it is forwarded to Verizon and uses their facilities to get it to the power company data center. Apparently, they use both cellular and wired depending on where the collector is and what facilities are available. Out where I am, they use cellular links to forward the data.
Did it have a pulse output? NO DARN IT. I was looking forward to setting up hardware to read the pulses to see how accurate my measurements are over the long haul. This was disappointing, but since I already measure the power myself, I don't have much to complain about.
So, I didn't gain much with the new meter. However the power company will collect data by usage time and I can prowl around their data and compare it to mine. If they show a significant difference, I might have to talk to them about it. Although, if they record less than I read, they won't be hearing from me.
What's fun though is that people have taken these meters apart and looked inside <link>. This article is absolutely great because it matches my meter exactly. I stole a couple of pictures from the article because I can't be sure it won't disappear at some point and I want to point out a couple of things:
This is the bottom plate of the meter. Notice the large wires to connect the supply to my house, they have a current transformer wrapped around them. This is almost exactly the way I measure power with my power monitor <link> The big difference is that I had to use two CTs to measure the power since the wires were too far apart to get one CT to go around them both. Nice to have some validation after so long. I've had to explain this a thousand times to various folk. Here's a picture of the CT with the wires after the author removed them from the housing:
Clever weren't they? The CT leads into a circuit board that measures the current and voltage to calculate the power usage. Darn, this is really starting to sound familiar.
There's a ton of conspiracy folk out there that see smart meters as a threat. They could become one at some point given an unrestricted power company and shoddy government oversight, but I'm not worried about that. I'm excited about the technology. This little guy can measure my power and send it up to APS (my power company) once it's there, I can grab it and do comparisons against my own measurements.
Heck, I'm using my power company as another cloud storage.
Tuesday, June 3, 2014
My Freezer and the Iris Smart Switch, Measuring Power
I went to Lowe's Home Improvement the other day to get supplies for jobs around the house that need to be done, you know caulk needing replacement, some spray foam, wire brush, the things we all have to get from time to time, but somehow, I wound up in front of the Iris display. Shame on me.
When I got home with the stuff I needed, and my new Iris smart (which was NOT on the list) I decided to hook it to my freezer so I can monitor both big appliances in the kitchen. Since I was lazy the last time I played with these devices, I had to modify the code to support more than one switch. That wasn't too hard. The switch connected first try and started reporting. I was a bit surprised at what I got out:
I labeled a couple of interesting things; remember I have separate refrigerator and freezer, this is the freezer only.
The spike and drop at point A is the automatic defrost running. The way these things work is there is an actual heating element on the evaporator in the freezer, and a timer runs it a couple or more times a day. The timer kicks on and the heater brings the evaporator up to 40°F when a thermostat on the evaporator coil opens and turns off the heater. The rest of the defrost cycle runs and control goes back to the temperature control system. Which usually means the compressor kicks on. This period of time allows the water to drain away into a pan where a fan evaporates it. My defrost cycle falls in the 'Peak' period of billing, so that's not good. The timer also runs on a little less than a 24 hour cycle, so it doesn't kick in at the same time each day.
This all means that I have a heater kicking on during the peak period that could be run some other time. Fortunately, there's a little slot on the defrost timer that allows it to be advanced such that the defrost time can be moved, but since it will eventually move back into the peak period, I'll have to readjust it from time to time. Notice it runs twice a day, that'll just complicate matters a bit. This little timer may be a good place to put an Arduino synced up to my house time for absolute control of the defrost cycle. No, I'm not getting too anal about this ...
Point B is the lights inside the darn thing. This freezer has 8 stinking lights in it; four on the top, and four on the bottom. Time to see if I can get a deal on led lights to replace these incandescents along with the ones in the fridge. I mean the lights actually use more power while they're on than the compressor does. Granted, they aren't on very often or for very long, but crap, close to 300 watts to light up the freezer?
Point C is the automatic ice maker. When ice is low, this thing kicks on periodically to roll the new cubes into the tray and opens a valve to let water in for more. This seemingly simple operation uses over 150 watts. This is also short lived, so it can be tolerated.
What surprised me was how long the compressor runs. I was worried that there may be something wrong since low coolant pressure or an air leak can cause the compressor to run too much. I went to the GE site and looked up their explanation of the compressor cycle and what I'm seeing is perfectly normal. The compressor on a freezer can run from 75 to 90 percent of the time under normal operation. This seems excessive, but the motor is designed to take this much use. It only uses 130 watts, so that's not too bad, and it does cycle somewhat, especially in the late evening early morning period.
What's becoming more apparent over time is the insignificance of parasitic power devices. Little things like cell phone chargers that are left plugged in, televisions that look for remote control signals, wireless phones, etc. I'll start checking on these devices, but it appears that all of them are roughly equivalent to a 60 watt light bulb. You'll get a lot of bang on the power bill by paying attention to the lights and ignoring the small devices around the house.
Now, I have to get the caulk gun and get back to what I'm supposed to be doing.
When I got home with the stuff I needed, and my new Iris smart (which was NOT on the list) I decided to hook it to my freezer so I can monitor both big appliances in the kitchen. Since I was lazy the last time I played with these devices, I had to modify the code to support more than one switch. That wasn't too hard. The switch connected first try and started reporting. I was a bit surprised at what I got out:
I labeled a couple of interesting things; remember I have separate refrigerator and freezer, this is the freezer only.
The spike and drop at point A is the automatic defrost running. The way these things work is there is an actual heating element on the evaporator in the freezer, and a timer runs it a couple or more times a day. The timer kicks on and the heater brings the evaporator up to 40°F when a thermostat on the evaporator coil opens and turns off the heater. The rest of the defrost cycle runs and control goes back to the temperature control system. Which usually means the compressor kicks on. This period of time allows the water to drain away into a pan where a fan evaporates it. My defrost cycle falls in the 'Peak' period of billing, so that's not good. The timer also runs on a little less than a 24 hour cycle, so it doesn't kick in at the same time each day.
This all means that I have a heater kicking on during the peak period that could be run some other time. Fortunately, there's a little slot on the defrost timer that allows it to be advanced such that the defrost time can be moved, but since it will eventually move back into the peak period, I'll have to readjust it from time to time. Notice it runs twice a day, that'll just complicate matters a bit. This little timer may be a good place to put an Arduino synced up to my house time for absolute control of the defrost cycle. No, I'm not getting too anal about this ...
Point B is the lights inside the darn thing. This freezer has 8 stinking lights in it; four on the top, and four on the bottom. Time to see if I can get a deal on led lights to replace these incandescents along with the ones in the fridge. I mean the lights actually use more power while they're on than the compressor does. Granted, they aren't on very often or for very long, but crap, close to 300 watts to light up the freezer?
Point C is the automatic ice maker. When ice is low, this thing kicks on periodically to roll the new cubes into the tray and opens a valve to let water in for more. This seemingly simple operation uses over 150 watts. This is also short lived, so it can be tolerated.
What surprised me was how long the compressor runs. I was worried that there may be something wrong since low coolant pressure or an air leak can cause the compressor to run too much. I went to the GE site and looked up their explanation of the compressor cycle and what I'm seeing is perfectly normal. The compressor on a freezer can run from 75 to 90 percent of the time under normal operation. This seems excessive, but the motor is designed to take this much use. It only uses 130 watts, so that's not too bad, and it does cycle somewhat, especially in the late evening early morning period.
What's becoming more apparent over time is the insignificance of parasitic power devices. Little things like cell phone chargers that are left plugged in, televisions that look for remote control signals, wireless phones, etc. I'll start checking on these devices, but it appears that all of them are roughly equivalent to a 60 watt light bulb. You'll get a lot of bang on the power bill by paying attention to the lights and ignoring the small devices around the house.
Now, I have to get the caulk gun and get back to what I'm supposed to be doing.
Monday, June 2, 2014
Just for Fun
I just answered an email from a nice person out there that was curious about my avatar that shows up in various places I post and my little home web site. It's a squirrel:
I love this game. I actually keep a Nintendo 64 around to play the game and occasionally drag it out to refresh my memory. What could keep the attention of an old man that must have outgrown silly little games a long time ago? Here's a link to one of the fun monsters in the game. It's a rather long video, but you'll see what attracted me to the game. <link>
I decided that Conker was my hero. This led me to use other squirrels from time to time as avatars, or just to illustrate a point:
Unless they come to my house.
Apple's Big Home Automation Announcement
A couple of days ago I ran across an article that said Apple was going to announce a movement into home automation. Every time Apple coughs, the news services go orgasmic and start showing everything they can, the tech copycat articles start popping up using text they stole from some other site, and the news aggregators (huffington and the like) duplicate the articles all over the place. I knew this was going to be fun. Here's a link to the article I ran into <link>
So, today was the day that Apple was going to make their earth-shattering announcement. The home automation part of it didn't really get out there until several hours latter, the copycats were too busy talking about the Beats purchase and maybe some new changes to iOS. But, I found one here <link>, I'll bet the audience of Appleophiles were going all big eyed at what was possible. Why, you can shut your garage door, set your thermostats, turn the lights on ... I actually laughed out loud.
For goodness sake, folks like me have been doing this for years now. We control our pool, measure the weather (inside and out), and even measure the level of our septic tanks; we mix and match devices, services, and web pages to our hearts content. We don't pay monthly subscriptions and are not at the mercy of a single supplier.
We are free to handle our house the way we want to.
I think it's a good thing that they are doing this because it won't be long before one of us hacks into the various devices and makes them work for us. This means more devices we can play with, more things we can automate, more of Apple's research we can leverage to our own ends. I don't think they think like I do though; they'll want a monthly fee for web services and get even richer. I'll be chuckling as I take apart their devices to see how they work.
Another good thing, Apple's announcement will open people's eyes to what is possible. Much more so than my little blog and the hundreds of others out there that do the same kind of thing. There's a whole lot of us that are doing it ourselves, and this may enlist more of us. Sure, we're outnumbered by the folk that just buy a solution, but where's the fun and satisfaction in that?
Meanwhile, I got a notice today that I'm going to get a smart meter on the house. Finally, the powers-that-be are moving technology out into my neck of the cactus plants. Now, I may have to build up a device to read the pulses from the meter and get power readings that are as accurate as theirs, but much more current than what they provide. I suspect I won't be able to crack into their monitoring signal, but I don't really need that. However, you know I'm going to try.
So, today was the day that Apple was going to make their earth-shattering announcement. The home automation part of it didn't really get out there until several hours latter, the copycats were too busy talking about the Beats purchase and maybe some new changes to iOS. But, I found one here <link>, I'll bet the audience of Appleophiles were going all big eyed at what was possible. Why, you can shut your garage door, set your thermostats, turn the lights on ... I actually laughed out loud.
For goodness sake, folks like me have been doing this for years now. We control our pool, measure the weather (inside and out), and even measure the level of our septic tanks; we mix and match devices, services, and web pages to our hearts content. We don't pay monthly subscriptions and are not at the mercy of a single supplier.
We are free to handle our house the way we want to.
I think it's a good thing that they are doing this because it won't be long before one of us hacks into the various devices and makes them work for us. This means more devices we can play with, more things we can automate, more of Apple's research we can leverage to our own ends. I don't think they think like I do though; they'll want a monthly fee for web services and get even richer. I'll be chuckling as I take apart their devices to see how they work.
Another good thing, Apple's announcement will open people's eyes to what is possible. Much more so than my little blog and the hundreds of others out there that do the same kind of thing. There's a whole lot of us that are doing it ourselves, and this may enlist more of us. Sure, we're outnumbered by the folk that just buy a solution, but where's the fun and satisfaction in that?
Meanwhile, I got a notice today that I'm going to get a smart meter on the house. Finally, the powers-that-be are moving technology out into my neck of the cactus plants. Now, I may have to build up a device to read the pulses from the meter and get power readings that are as accurate as theirs, but much more current than what they provide. I suspect I won't be able to crack into their monitoring signal, but I don't really need that. However, you know I'm going to try.
Subscribe to:
Posts (Atom)