Saturday, January 10, 2015

Wemo Light Switches, a Completely New Approach

I have several posts on implementing the Wemo switches into my house <link, link, link> and of course a couple of other modifications to make things a little easier; lastly, where I finally found out what was causing them to just go away from time to time <link>.  Even with all this attention, the things still are an aggravation.  But not just for me; one of my readers, Glenn Tracy, has several of the switches, and one Wemo Insight plug that have been driving him nuts.  Glenn made the serious mistake of mentioning arp, so I enlisted him to try the various ideas out as I coded them (he probably regrets that now), and I started exploring possibilities.

The devices pretty much stay on line since I found a cause of the intermittent failure, but then they get hard to find because they don't respond well to Upnp discovery messages.  Since the IP address is assigned by DHCP, I can't rely on that to find them, plus the port they respond to can change any time the switch feels cranky.  So how the heck can I find them reliably ... arp seemed to work.  TCP/IP relies on the arp protocol to communicate, that should be reliable enough.  So, how to use it?

Fortunately, someone else has addressed this problem with a couple of tools, arping and arp-scan.  Arping relies on some knowledge in advance, but arp-scan doesn't.  You can issue an arp-scan on your home network and every machine on the network will be found.  Naturally, this isn't perfect, some devices guard against response to prevent hacking, and some devices don't respond on the first interaction.  But, I'm looking for Wemo switches, they respond, so now I have to figure out how to get the port number that the switch currently listens to.

Taking a chapter from my hacks on various other switches and devices, I decided to just try all the ports and see which one worked.  Since the switches I have only use ports 49152, 49153, and 49154, I can try all of them in a matter of seconds and not worry about it taking a ton of time searching.  If it turns out later that they go higher from time to time, I can simply add another port to the list.  Now, armed with this possibility, I tried out arp-scan.  Arp-scan is a normal linux tool, so all we have to do is install it:

sudo apt-get install arp-scan

And let the installation run.  Now we have to make it run as a root process because raw sockets need root permission to be effective:

sudo chmod +s /usr/bin/arp-scan

That will give the executable the 'suid' bit and since it's owned by root, it will get all the capabilities of a root process.  Nice, now run it and see what happens:

 arp-scan -q -v 192.168.0.0-192.168.0.255
Interface: eth0, datalink type: EN10MB (Ethernet)
Starting arp-scan 1.8.1 with 256 hosts (http://www.nta-monitor.com/tools/arp-scan/)
192.168.0.1     40:4a:03:de:41:87
192.168.0.2     00:1d:7e:91:12:0a
192.168.0.28    ec:1a:59:ce:82:71
192.168.0.34    4c:82:cf:83:c5:05
192.168.0.45    ec:e0:9b:b9:31:11
192.168.0.50    00:1d:7e:91:12:0a
192.168.0.29    ec:1a:59:ce:7c:8d
192.168.0.44    78:4b:87:41:5d:53
192.168.0.43    ec:1a:59:cd:99:55
192.168.0.22    2c:d0:5a:f6:dc:65
192.168.0.26    ec:1a:59:e8:fe:81
192.168.0.47    18:22:7e:d2:04:a0
192.168.0.202   de:ad:be:ef:fe:ed
192.168.0.203   de:ad:be:ef:fe:ee
192.168.0.204   de:ad:be:ef:fe:ef
---     Pass 1 complete
---     Pass 2 complete

18 packets received by filter, 0 packets dropped by kernel
Ending arp-scan 1.8.1: 256 hosts scanned in 1.258 seconds (203.50 hosts/sec). 15 responded

Notice a few things: I carefully chose the options to give me the information I needed while excluding the built-in capability of finding the manufacturer of the devices found by table lookup.  I was only interested in Belkin devices, and the table lookup depends on having up to date tables.  I really don't want to have to worry about keeping some table updated, so I'll handle that in the code for controlling the switch.  Also, arp-scan will take a range of addresses; since I limit my DHCP devices to the range 0-50, I should be able to scan all addresses in a few seconds.  That should be much, much faster than waiting for devices to respond to the (often ineffective) upnp discovery command.

The devices have been found, and a little code can isolate out the Belkin devices, so now it's time to find the port number the switch listens to.  This serves another purpose, to separate the switches from other Belkin devices that may show up over time.  It wouldn't do much good to tell a router or ethernet switch to turn off thinking it was a switch.  We've already discovered many intimate details of the switches including the location of the switch's setup xml description.  So a simple HTML request to switch-ip-address:possible-port/setup.xml should return a description of the switch.  Once we have the html in hand, just look inside it for the keywords and we know what we have.  I'm only interested in an Insite or a lightswitch; the other devices can wait for a later hack.  If nothing comes back, then I have the wrong port, just try the next one.  Here's an example of the setup.xml output that I sucked out of one of my switches:

<root xmlns="urn:Belkin:device-1-0"> <specVersion> <major>1</major> <minor>0</minor> </specVersion> <device> <deviceType>urn:Belkin:device:lightswitch:1</deviceType> <friendlyName>outsidegarage</friendlyName> <manufacturer>Belkin International Inc.</manufacturer> <manufacturerURL>http://www.belkin.com</manufacturerURL> <modelDescription>Belkin Plugin Socket 1.0</modelDescription> <modelName>LightSwitch</modelName> <modelNumber>1.0</modelNumber> <modelURL>http://www.belkin.com/plugin/</modelURL> <serialNumber>221332K130012B</serialNumber> <UDN>uuid:Lightswitch-1_0-221332K130012B</UDN> <UPC>123456789</UPC> <macAddress>EC1A59E8FE80</macAddress> <firmwareVersion>WeMo_WW_2.00.6395.PVT</firmwareVersion> <iconVersion>3|49153</iconVersion> <binaryState>0</binaryState> <iconList> <icon> <mimetype>jpg</mimetype> <width>100</width> <height>100</height> <depth>100</depth> <url>icon.jpg</url> </icon> </iconList> <serviceList> <service> <serviceType>urn:Belkin:service:WiFiSetup:1</serviceType> <serviceId>urn:Belkin:serviceId:WiFiSetup1</serviceId> <controlURL>/upnp/control/WiFiSetup1</controlURL> <eventSubURL>/upnp/event/WiFiSetup1</eventSubURL> <SCPDURL>/setupservice.xml</SCPDURL> </service> <service> <serviceType>urn:Belkin:service:timesync:1</serviceType> <serviceId>urn:Belkin:serviceId:timesync1</serviceId> <controlURL>/upnp/control/timesync1</controlURL> <eventSubURL>/upnp/event/timesync1</eventSubURL> <SCPDURL>/timesyncservice.xml</SCPDURL> </service> <service> <serviceType>urn:Belkin:service:basicevent:1</serviceType> <serviceId>urn:Belkin:serviceId:basicevent1</serviceId> <controlURL>/upnp/control/basicevent1</controlURL> <eventSubURL>/upnp/event/basicevent1</eventSubURL> <SCPDURL>/eventservice.xml</SCPDURL> </service> <service> <serviceType>urn:Belkin:service:firmwareupdate:1</serviceType> <serviceId>urn:Belkin:serviceId:firmwareupdate1</serviceId> <controlURL>/upnp/control/firmwareupdate1</controlURL> <eventSubURL>/upnp/event/firmwareupdate1</eventSubURL> <SCPDURL>/firmwareupdate.xml</SCPDURL> </service> <service> <serviceType>urn:Belkin:service:rules:1</serviceType> <serviceId>urn:Belkin:serviceId:rules1</serviceId> <controlURL>/upnp/control/rules1</controlURL> <eventSubURL>/upnp/event/rules1</eventSubURL> <SCPDURL>/rulesservice.xml</SCPDURL> </service> <service> <serviceType>urn:Belkin:service:metainfo:1</serviceType> <serviceId>urn:Belkin:serviceId:metainfo1</serviceId> <controlURL>/upnp/control/metainfo1</controlURL> <eventSubURL>/upnp/event/metainfo1</eventSubURL> <SCPDURL>/metainfoservice.xml</SCPDURL> </service> <service> <serviceType>urn:Belkin:service:remoteaccess:1</serviceType> <serviceId>urn:Belkin:serviceId:remoteaccess1</serviceId> <controlURL>/upnp/control/remoteaccess1</controlURL> <eventSubURL>/upnp/event/remoteaccess1</eventSubURL> <SCPDURL>/remoteaccess.xml</SCPDURL> </service> <service> <serviceType>urn:Belkin:service:deviceinfo:1</serviceType> <serviceId>urn:Belkin:serviceId:deviceinfo1</serviceId> <controlURL>/upnp/control/deviceinfo1</controlURL> <eventSubURL>/upnp/event/deviceinfo1</eventSubURL> <SCPDURL>/deviceinfoservice.xml</SCPDURL> </service> <service> <serviceType>urn:Belkin:service:smartsetup:1</serviceType> <serviceId>urn:Belkin:serviceId:smartsetup1</serviceId> <controlURL>/upnp/control/smartsetup1</controlURL> <eventSubURL>/upnp/event/smartsetup1</eventSubURL> <SCPDURL>/smartsetup.xml</SCPDURL> </service> <service> <serviceType>urn:Belkin:service:manufacture:1</serviceType> <serviceId>urn:Belkin:serviceId:manufacture1</serviceId> <controlURL>/upnp/control/manufacture1</controlURL> <eventSubURL>/upnp/event/manufacture1</eventSubURL> <SCPDURL>/manufacture.xml</SCPDURL> </service> </serviceList> <presentationURL>/pluginpres.html</presentationURL> </device> </root>

The tag <modelName>LightSwitch</modelName> tells me that this is a light switch, as opposed to an Insight, which would look like <modelName>Insight</modelName>.  This matters because the Insight switch returns a different value when it's turned on. The MAC address is in there, and a lot of other things that are used during operation.  For example to find the friendly name (the name assigned by the user) look at the tag <friendlyName>outsidegarage</friendlyName>.

The very fact that I got the XML back means I found the port and the tags tell me what it is, so now I have enough information, just some code to control the switches and I should be done.  Yeah, right.  When my victim er ... uh ... partner Glenn in this effort tested it, there were a number of problems that came up.  I had to put the IP address range in the .houserc file because there just wasn't any way to reliably discover the network range we were interested in.  Then, we found out that the wemo switches took to long to respond to the arp request and it had to be retried, sometimes several times to get them to talk.  When the switches decide to change addresses, there has to be a rediscovery to get the new port number.  You know, the usual list of things that will drive you nuts using a device differently than it was intended.

But, we overcame the ton of 'glitches' that almost appear to have been designed into the switch to keep people like me from doing things like this, and in the process made it much faster, much more reliable, and able to leap tall buildings with a single bound.  There are some caveats though:  The switches can still disappear.  Yes, even with a different method of discovery, and port change retries, the things will fail to respond long enough to evade detection.  To overcome this, the control process will simply exit after a few retries, and some controlling tool can automatically restart it to do the entire discovery process all over again.  This will find the switches and set them up to be used all over again.  A simple script like this:

#!/bin/bash
while : do         echo "Starting wemoccontrol"         wemocontrol.py         sleep 10 done

can restart the process.  Yes, you have to replace the commands with something suitable to your environment, and the sleep is to keep it under control in case you make a mistake, but you can use this to test the idea.  I use upstart and a config file to accomplish this for all the processes I run on the Pi, why invent something new when a great tool already exists?

So, instead of using upnp discovery, I switched to arp discovery, I find the Belkin device, and among those I try the various ports to discover Wemo light switches and Insite devices, then I bring up a CherryPy server to control them.  When they fail to respond to a request, I try three more times to make them respond and then give up and let the process die to be restarted by some other control software.  The controls are the same as my previous tries, I record the state in a Sqlite3 database and that in turn can be read by anything running on the machine.  Sounds pretty complicated, but it's much, much simpler than the other techniques I came up with.  This is very lightweight and works quite a bit faster than the others, and so far, much more reliably.  Here's the code, and it has also been placed in Github to make it easier to grab if you want <link>.

#! /usr/bin/python
# Checking Wemo Switches
#
import subprocess
import commands
from datetime import datetime, timedelta
import time
import urllib2
import BaseHTTPServer
from socket import *
import sys
import json
import re
import argparse
import sqlite3
import cherrypy
from houseutils import lprint, getHouseValues, timer, checkTimer

#--------This is for the HTML interface 
def openSite(Url):
    #lprint (Url)
    webHandle = None
    try:
        webHandle = urllib2.urlopen(Url, timeout=2) # give up in 2 seconds
    except urllib2.HTTPError, e:
        errorDesc = BaseHTTPServer.BaseHTTPRequestHandler.responses[e.code][0]
        #print "Error: (opensite) cannot retrieve URL: " + str(e.code) + ": " + errorDesc
        raise
    except urllib2.URLError, e:
        #print "Error: (openSite) cannot retrieve URL: " + e.reason[1]
        raise
    except:  #I kept getting strange errors when I was first testing it
        e = sys.exc_info()[0]
        #print ("(opensite) Odd Error: %s" % e )
        raise
    return webHandle

def talkHTML(ip, command):
    website = openSite("HTTP://" + ip + '/' + urllib2.quote(command, safe="%/:=&?~#+!$,;'@()*[]"))
    # now (maybe) read the status that came back from it
    if website is not None:
        websiteHtml = website.read()
        return  websiteHtml
        
# and this is for the SOAP interface        
# Extract the contents of a single XML tag from the data
def extractSingleTag(data,tag):
    startTag = "<%s" % tag
    endTag = "</%s>" % tag

    try:
        tmp = data.split(startTag)[1]
        index = tmp.find('>')
        if index != -1:
            index += 1
            return tmp[index:].split(endTag)[0].strip()
    except:
        pass
    return None

def sendSoap(actionName, whichOne, actionArguments):
    argList = ''
    soapEnd = re.compile('<\/.*:envelope>')
    if not actionArguments:
        actionArguments = {}
    for item in switches:
        if item["name"] == whichOne:
            thisOne = item
            break;
    switchIp = item["ip"]
    switchPort = item["port"]
    
    for arg,(val,dt) in actionArguments.iteritems():
        argList += '<%s>%s</%s>' % (arg,val,arg)

    soapRequest = 'POST /upnp/control/basicevent1 HTTP/1.1\r\n'
    # This is the SOAP request shell, I stuff values in it to handle
    # the various actions 
    # First the body since I need the length for the headers
    soapBody =  '<?xml version="1.0"?>\n'\
            '<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">\n'\
            '<SOAP-ENV:Body>\n'\
            '\t<m:%s xmlns:m="urn:Belkin:service:basicevent:1">\n'\
            '%s\n'\
            '\t</m:%s>\n'\
            '</SOAP-ENV:Body>\n'\
            '</SOAP-ENV:Envelope>' % (actionName,argList,actionName)

    #These are the headers to send with the request
    headers =   {
            'Host':'%s:%s' % (switchIp, switchPort),
            'Content-Length':len(soapBody),
            'Content-Type':'text/xml',
            'SOAPAction':'"urn:Belkin:service:basicevent:1#%s"' % (actionName)
            }
    #Generate the final payload
    for head,value in headers.iteritems():
        soapRequest += '%s: %s\r\n' % (head,value)
    soapRequest += '\r\n%s' % soapBody
    if showXml:
        print stars
        print "***REQUEST"
        print soapRequest
 
    try:
        sock = socket(AF_INET,SOCK_STREAM)
        sock.connect((switchIp,int(switchPort)))
        sock.settimeout(3);  # don't want to hang forever, ever
        sock.send(soapRequest)
        soapResponse = ""
        while True:
            data = sock.recv(1024)
            if not data:
                break
            else:
                soapResponse += data
                if soapEnd.search(soapResponse.lower()) != None:
                    break
        if showXml:
            print "***RESPONSE"
            print soapResponse
            print stars
            print ''
        sock.close()
        (header,body) = soapResponse.split('\r\n\r\n',1)
        if not header.upper().startswith('HTTP/1.') and ' 200 ' in header.split('\r\n')[0]:
            print 'SOAP request failed with error code:',header.split('\r\n')[0].split(' ',1)[1]
            errorMsg = self.extractSingleTag(body,'errorDescription')
            if errorMsg:
                print 'SOAP error message:',errorMsg
            return None
        else:
            return body
    except Exception, e:
        lprint ('Caught exception in sending:', e, switchIp, switchPort)
        sock.close()
        return None
    except KeyboardInterrupt:
        print "Keyboard Interrupt"
        sock.close()
        return None

# This will look at the result from sendSoap, and if the
# switch disappeared, it will try and get the new port number
# and update the various items.  This should allow the code 
# to continue as if the switch never decided to change its
# port number
def sendCommand(actionName, whichOne, actionArguments):
    result = sendSoap(actionName, whichOne, actionArguments)
    if result is not None:
        return result
    # it failed, now we have to do something about it
    # first, get the switch entry to check for a port change
    for item in switches:
        if item["name"] == whichOne:
            thisOne = item
            break;
    switchIp = item["ip"]
    switchPort = item["port"]
    # try to get the port number from the switch a few times
    for i in range(0,3): # Only try this three times
        lprint ("Trying to recover the switch %s"%whichOne)
        # getPort doesn't use sendSoap, so this call won't recurs
        newEntry = getPort(switchIp)
        # if the port changed, try and get the new one
        if newEntry is not None:
            # fine, it's at least alive, grab the port number,
            # print something, and and stuff it in the database
            # if it didn't change this won't break it, but if 
            # it did change, this will fix it.
            item["port"] = newEntry["port"]
            lprint ("Switch", whichOne, "changed ip from", switchPort, "to", newEntry["port"])
            dbconn = sqlite3.connect(DATABASE)
            c = dbconn.cursor()
            try:
                c.execute("update lights " 
                    "set port=? where name = ?;",
                    (newEntry["port"], whichOne))
            except sqlite3.OperationalError:
                lprint("Database is locked, record skipped")
            dbconn.commit()
            dbconn.close()
            # now try the command again
            # if it died completely it may have come back by now,
            # or if the port changed, this will try it one more time
            # it needs a limit this because this call will recurs
            result = sendSoap(actionName, whichOne, actionArguments)
            if result is not None:
                lprint("Switch recovered")
                return result
            time.sleep(1) #give the switch time to catch its breath
        else: 
            # this means the switch is not responding to HTML
            # so try the getPort again to see if it's back yet
            # There's no point in sending the soap command yet
            time.sleep(1) #give the switch time to catch its breath
            continue
    # it failed three times, just give up, die and let the system
    # restart the process.
    exit("The switch %s went away"% whichOne)
        
        
# Step through each light and see get its current state
# then record the state in the database.
def doLights():
    for switch in switches:
        thisOne = switch['name']
        updateDatabase(thisOne,get(thisOne))

def keepAlive():
    '''
    I update the database periodically with the time so I can check to see 
    if things are holding together.  I currently use the time in the light 
    switch records for this.
    '''
    lprint(" keep alive")
    for switch in switches:
        thisOne = switch['name']
        updateDatabase(thisOne, get(thisOne), force=True)

        
def get(whichone):
    ''' 
    Returns On or Off
    '''
    resp = sendCommand('GetBinaryState', whichone, {})
    if resp is not None:
        tagValue = extractSingleTag(resp, 'BinaryState').split('|')[0]
        return 'Off' if tagValue == '0' else 'On'
    return 'Off'

def on(whichone):
    """
    BinaryState is set to 'Error' in the case that it was already on.
    """
    resp = sendCommand('SetBinaryState', whichone, {'BinaryState': (1, 'Boolean')})
    if resp is not None:
        tagValue = extractSingleTag(resp, 'BinaryState').split('|')[0]
        status = 'On' if tagValue in ['1', '8', 'Error'] else 'Off'
        handleUpdate(whichone, status)
        lprint("turned %s on"%(whichone))
        return status
    return 'Off'

def off(whichone):
    """
    BinaryState is set to 'Error' in the case that it was already off.
    """
    resp = sendCommand('SetBinaryState', whichone, {'BinaryState': (0, 'Boolean')})
    if resp is not None:
        tagValue = extractSingleTag(resp, 'BinaryState').split('|')[0]
        status = 'Off' if tagValue in ['0', 'Error'] else 'On'
        handleUpdate(whichone, status)
        lprint("turned %s off"%(whichone))
        return status
    return 'Off'
    
def toggle(whichOne):
    if (get(whichOne) == 'On'):
        off(whichOne)
    else:
        on(whichOne)
        
def outsideLightsOn():
    lprint (" Outside lights on")
    on("outsidegarage")
    on("frontporch")
    on("cactusspot")
    
def outsideLightsOff():
    lprint (" Outside lights off")
    off("outsidegarage")
    off("frontporch")
    off("cactusspot")

        
def handleUpdate(whichone, status):
    for i in switches:
        if i['name'] == whichone:
            i['status'] = status
    updateDatabase(whichone, status)
    
def updateDatabase(whichone, status, force=False):
    ''' 
    This is running on a Pi and is not event driven, so polling like
    this will result in considerable wear to the SD card.  So, I'm going to 
    read the database to see if it needs to be changed before I change it.  
    According to everything I've read, reads are free, it's the writes that
    eat up the card.
    '''
    dbconn = sqlite3.connect(DATABASE)
    c = dbconn.cursor()
    c.execute("select status from lights where name = ?;",
        (whichone,))
    oldstatus = c.fetchone()
    if oldstatus[0] != status or force == True:
        lprint ("Had to update database %s, %s"%(whichone, status))
        try:
            c.execute("update lights " 
                "set status = ?, utime = ? where name = ?;",
                (status, time.strftime("%A, %B, %d at %H:%M:%S"), whichone))
            dbconn.commit()
        except sqlite3.OperationalError:
            lprint("Database is locked, record skipped")
    dbconn.close()

# If a command comes in from somewhere, this is where it's handled.
def handleCommand(command):
    lprint(str(command))
    # the command comes in from php as something like
    # ('s:17:"AcidPump, pumpOff";', 2)
    # so command[0] is 's:17:"AcidPump, pumpOff'
    # then split it at the "  and take the second item
    try:
        c = str(command[0].split('\"')[1]).split(',')
    except IndexError:
        c = str(command[0]).split(' ')    #this is for something I sent from another process
    lprint(c)
    if (c[0] == 'OutsideLightsOn'):
        outsideLightsOn()
    elif (c[0] == 'OutsideLightsOff'):
        outsideLightsOff()
    elif (c[0] == 'fPorchToggle'):
        toggle("frontporch")
    elif(c[0] == 'garageToggle'):
        toggle("outsidegarage")
    elif (c[0] == 'cactusToggle'):
        toggle("cactusspot")
    elif (c[0] == 'patioToggle'):
        toggle("patio")
    else:
        lprint("Weird command = " + str(c))

# First the process interface, it consists of a status report and
# a command receiver.
class WemoSC(object):
    @cherrypy.expose
    @cherrypy.tools.json_out() # This allows a dictionary input to go out as JSON
    def status(self):
        status = []
        for item in switches:
            status.append({item["name"]:get(item["name"])})
        return status
        
    @cherrypy.expose
    def pCommand(self, command):
        handleCommand((command,0));
        
    @cherrypy.expose
    def index(self):
        status = "<strong>Current Wemo Light Switch Status</strong><br /><br />"
        for item in switches:
            status += item["name"] +" is " + get(item["name"]) + "&nbsp;&nbsp;"
            status += '<a href="wemocommand?whichone='+item["name"]+'"><button>Toggle</button></a>'
            status += "<br />"
        return status
        
    @cherrypy.expose
    def wemocommand(self, whichone):
        # first change the light state
        toggle(whichone)
        # now reload the index page to tell the user
        raise cherrypy.InternalRedirect('/index')

# given the ip of a Belkin device this will try the ports that
# are used on the Wemo switches to see which one works.  The assumption
# is that if none of the ports work, it's not a switch, it's a modem or
# something else.
def getPort(ip):
    entry = []
    for p in ["49153", "49154", "49155"]:
        try:
            resp = talkHTML(ip + ':' + p + "/setup.xml", "")
            if debug:
                print "\tfound one at", b[0], "port", p
            if showXml:
                print stars
                print "response from switch"
                print resp
                print stars
            name = extractSingleTag(resp, 'friendlyName')
            model = extractSingleTag(resp, 'modelName')
            entry = {"mac":b[1],"ip":b[0], "port":p, "name":name, "model":model}
            return entry
        except timeout:
            continue
        except urllib2.URLError:
            continue
        except:
            e = sys.exc_info()[0]
            print ("Unexpected Error: %s" % e )
            continue
    return None
        
####################### Actually Starts Here ################################    
debug = False
showXml = False
if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("-d", "--debug",
        action = "store_true",
        help='debug flag')
    parser.add_argument("-x", "--xml",
        action = "store_true",
        help='show xml')
    parser.add_argument('count',type=int);
    args = parser.parse_args()
    if args.debug:
        print "Running with debug on"
        debug = True
    if args.xml:
        print "Running with showXML on"
        showXml = True
    targetNumber = args.count

    stars = "*********************************************"

    #-------------------------------------------------
    # the database where I'm storing stuff
    DATABASE=getHouseValues()["database"]
    lprint("Using database ", DATABASE);
    # Get the ip address and port number you want to use
    # from the houserc file
    ipAddress=getHouseValues()["wemocontrol"]["ipAddress"]
    port = getHouseValues()["wemocontrol"]["port"]
    lprint("started looking for {} switches".format(targetNumber))

    # This works on my machine, but you may have to mess with it
    # The arp-scan below tells it not to look up the manufacturer because I
    # didn't want to worry about keeping the tables that are used up to date,
    # the -l tells it to find the local net address on its own, and 
    # -v (verbose) will print that net address so I can show it for debugging
    # I take the scan range out of the .houserc file, it's an entry under wemocontrol
    # that looks like "scanRange":"192.168.0.1-192.168.0.50" adjust this as
    # needed
    try:
        scanRange = getHouseValues()["wemocontrol"]["scanRange"]
        arpCommand = "arp-scan -q -v %s 2>&1" %(scanRange)
    except KeyError:
        print "No entry in .houserc for wemocontrol scanRange"
        exit();

    while True:
        devices = [];
        # first the devices on the network
        if debug:
            print "arp-scan command is:", arpCommand
        theList = subprocess.check_output(arpCommand,shell=True);
        # split the output of the arp-scan into lines instead of a single string
        lines = theList.splitlines()
        # this looks at each line and grabs the addresses we're interested in
        # while ignoring the lines that are just information.
        for line in lines:
            allowedDigits = set("0123456789abcdef:. \t")
            if all(c in allowedDigits for c in line):
                d = line.split()
                try:
                    devices.append([d[0], d[1]])
                except IndexError: # an empty line will pass the test
                    continue
        # arp-scan can give the same addresses back more than once
        # step through the list and remove duplicates
        temp = []
        for e in devices:
            if e not in temp:
                temp.append(e)
        devices = temp
        if debug:
            print devices
        # for each device, look up the manufacturer to see if it was registered
        # to belkin
        bDevices = []
        # I got this list direct from the IEEE database and it may
        # need to be updated in a year or two.
        belkinList = ("001150", "00173F", "001CDF", "002275", "0030BD", 
                        "08863B", "94103E", "944452", "B4750E", "C05627", "EC1A59")
        for d in devices:
            if d[1].replace(':','')[0:6].upper() in belkinList:
                    bDevices.append([d[0],d[1]])
        if debug:
            print "These are the Belkin devices on the network"
            print bDevices
        if len(bDevices) < targetNumber: 
            lprint ("Only found", len(bDevices), "Belkin devices, retrying")
            time.sleep(1)
            continue
        # Got all that were asked for, continue to the next step
        
        # Now that we have a list of the Belkin devices on the network
        # We have to examine them to be sure they are actually switches
        # and not a modem or something else.  This will also assure that 
        # they will actually respond to a request.  They still may not work,
        # but at least we have a chance.
        switches = []
        for b in bDevices:
            result = getPort(b[0])
            if result is not None:
                switches.append(result)
        # Did we find enough switches ?
        if len(switches) < targetNumber: 
            lprint ("Only found", len(switches), "of them, retrying")
            devices = []
            continue
        # Yes we did, break out.
        break;
    # Now I'm going to check the database to see if it has been
    # adjusted to hold all the items (older version didn't have
    # ip, port, and mac addresses
    dbconn = sqlite3.connect(DATABASE)
    c = dbconn.cursor()
    c.execute("pragma table_info(lights);")
    dbrow = c.fetchall()
    if not any('ip' and 'mac' and 'port' in r for r in dbrow):
        lprint ("Database needs to be adjusted")
        lprint ("to hold ip, port, and MAC")
        try:
            print "adding ip if needed"
            c.execute("alter table lights add column ip text;")
        except sqlite3.OperationalError:
            print "ip was already there"
        try:
            print "adding mac if needed"
            c.execute("alter table lights add column mac text;")
        except sqlite3.OperationalError:
            print "mac was already there"
        try:
            print "adding port if needed"
            c.execute("alter table lights add column port text;")
        except sqlite3.OperationalError:
            print "port was already there"
        dbconn.commit()
    else:
        lprint ("Database already adjusted")
    for item in switches:
        try:
            c.execute("update lights " 
                "set ip = ?, mac=?, port=?where name = ?;",
                (item["ip"], item["mac"], item["port"], item["name"]))
            dbconn.commit()
        except sqlite3.OperationalError:
            lprint("Database is locked, record skipped")
    dbconn.commit()
    dbconn.close()
    lprint ("")
    lprint ("The list of", len(switches), "switches found is")
    for item in switches:
        lprint ("Friendly name:", item["name"])
        lprint ("Model:", item["model"])
        lprint ("IP address:", item["ip"])
        lprint ("Port number:", item["port"])
        lprint ("MAC:", item["mac"])
        lprint ('')
        
    # timed things.
    checkLightsTimer = timer(doLights, seconds=2)
    keepAliveTimer = timer(keepAlive, minutes=4)
    # Now configure the cherrypy server using the values
    cherrypy.config.update({'server.socket_host' : ipAddress.encode('ascii','ignore'),
                            'server.socket_port': port,
                            'engine.autoreload.on': False,
                            })
    # Subscribe to the 'main' channel in cherrypy with my timer
    cherrypy.engine.subscribe("main", checkTimer.tick)
    lprint ("Hanging on the wait for HTTP message")
    # Now just hang on the HTTP server looking for something to 
    # come in.  The cherrypy dispatcher will update the things that
    # are subscribed which will update the timers so the light
    # status gets recorded.
    cherrypy.quickstart(WemoSC())
    
    sys.exit("Told to shut down");

Notice that it is self contained.  It doesn't require Miranda.py anymore, nor any of the Ouimeaux libraries.  That's what makes it appealing to me; it's nice to have everything in one module and reasonably easy to understand.  I added a few parameters:

wemocontrol.py -h
usage: wemocontrol.py [-h] [-d] [-x] count

positional arguments:
  count

optional arguments:
  -h, --help   show this help message and exit
  -d, --debug  debug flag
  -x, --xml    show xml

I start it with wemocontrol.py 4, because I have four switches.  It will keep looking until it finds all four switches which could be a problem if one of them is disconnected, but I'll deal with that problem as it happens.  The -x parameter displays the SOAP messages as they are sent and received.  This can be used to find problems in the protocol or to understand how it works.  The -d is for debugging when you can't get things to work.  If you turn them both on, the volume of output is huge, so be prepared for it.  The -h is because I can never remember this stuff and wanted a reminder.  To run it, you must have an entry in a file called .houserc in the home directory of the user.  I use this file for all the entries for my processes.  This way I can keep things like keys, directory paths, etc in one place easily found.  The entries I have look like this:

"wemocontrol":{
"ipAddress":"192.168.0.205",
"port": 51001,
"scanRange":"192.168.0.1-192.168.0.50"},

I have a complete description of the file and how to use it here <link>.

Now, I must thank Glenn for his help in this effort.  This thing would have been a bear to test without him.  The rest of you, have fun with it.

1 comment:

  1. What a way to spend one's Christmas vacation. To be honest it was really fun and enlightening. I wanted to share a couple of my personal comments as well. We must have tested at least 12 versions of the code to get here. What we came up with though is really great. It's fast, easy to understand, stand alone, and best of all sits there and just works. I have yet to get upstart going, I just use the little script Dave wrote above, and my switches, when they disappear, are found again almost immediately. I have four WEMO light switches and one INSIGHT and to be able to control them with opensource code is vonderbar..... There is a bunch more work necessary on the insight plug to be able to get the usage data out of it, but for now I'll be using the IRIS switches and put the Insight plugs into the ON/OFF process. I have a totally different network than Dave's. Mine is in the 10.10.x.x address. Having mine in a different address space forced Dave to make the code more universal and with the .houserc file it has accomplished that. I really like what Dave has done with the code. Now this is a stand alone component and can be run, modified and updated without interfering with the house monitor code yet can update the SQLite3 database independently. This is turning out to be an excellent architecture for house automation. One final point. I have a barn that I am outfitting with a RaspberryPI and it will have a couple of WeMo switches in it. Monitoring and controlling them will now be a piece of cake.

    Glenn.

    ReplyDelete