Saturday, September 10, 2016

A New Web Interface for Desert-Home

Remotely monitoring and controlling my house has been part of the technical work on the house ever since I started. Like every other homeowner in the world I wonder if I closed the garage door ten miles away from the house. Monitoring the power usage and closing the garage doors was my main motivation for getting into this.

I came up with a web page that let me control things and see what was going on a few years ago, and have never been totally satisfied with it <link>. It worked and told me what I wanted, but it looks too small on a phone and is kinda clunky.

Here it is on my phone:



Later, I discovered the MIT Android App Inventor and created an app using it. This tool was (almost) exactly what I needed to get a native Android app on my phone<link>. The problem with it came up when the app got big. The way they work it is with a single work space that you place graphic elements into creating an application. When it gets big, you can't see enough of it to move things around and connect the dots. While I used that app, I looked around for other solutions.

This actually led me to create a native Android application that could talk to the house. The Android app was pretty cool:


Normally, the 'Wait' would be replaced with the current temp, and the various gauges would be showing data, but things changed (as usual) and I didn't update it. Obviously, I was really into green and brown at the time, plus I still liked the brass gauges. This app was fun because each of the device panels would slide in from the side and slide back off when I wanted to get rid of them. That left the display of only the items I was interested in at that moment. People were properly impressed; I was proud.

The problem is that Android is a pain to program to. The development environment is HUGE and takes a significant hunk of the storage on my laptop. Testing it means putting it on my phone, testing a bit, fixing it, putting it back on the phone, over and over again. The methods work, but they get really monotonous. There's an emulator in the SDK, but I couldn't get it to work reliably or fast enough to actually do things. Plus, at that time, the emulator couldn't talk over the network which means large portions of the code couldn't work there.

I'm not knocking the Android SDK, they did a fantastic job of putting it together and working out a way to develop apps, but it's a huge mess for an individual trying to do a simple job like this. I was taking on the jobs of a team of developers instead of just coming up with something to control my house.

I went looking again. Since I had come up with three methods already, I knew more about what I wanted to have:

A web interface of some kind that would work nicely on the phone as well as my laptop. I was perfectly willing to accept some compromises because of the difference in screen dimensions and mouse vs. finger input methods, but I wanted it to be mostly seamless between the two.

One set of code for both of them. It is too cumbersome for an individual bringing up a control system to keep track of code for different viewing devices. I could never remember to make corresponding changes to each piece, and one of them, or the other, always fell behind.

I want to be able to put a tablet on the wall, walk up to it, change something around the house and not think about what device I'm using to make the change. So, a phone, a tablet or two, the laptop, my friends computer, etc. Kind of daunting when you think about it; that's why it's been years of messing around.

But, I recently bought a Samsung Galaxy Note 7. Yes, the one that is exploding and catching fire and isn't allowed to be turned on when you're flying. That one. I love it. Sure it's big, but that means I can actually see things on it, and the fast charging is absolutely great. I can charge it from totally dead to full in less than two hours. No wonder they tend to blow up. Anyway, the new phone prompted me to take another look at something to control the house with.

Prowling around the web, I found several interfaces that looked promising, but they all had those esoteric marketing descriptions that have a lot of words, but very little to tell you how to actually use it. Some really good authoring systems that could fit the bill, but there were flexibility concerns with them. While I was looking, I kept thinking about how things that some corporation supplies change whenever they change a manager, so I decided to use some JavaScript library that I could, if needed, copy to my own machine in total and avoid the dependency on them keeping their stuff online and active. I do this with Arduino libraries, why not be prepared to do it with other stuff as well?

Enter JQuery Mobile. It's a couple of libraries that sit on top of the JQuery we all know and love (hate), and has been around for a few years. It's not maintained very much anymore since the lead developer has moved on, but it does what I want and handles all the crap of resizing various things when it displays on a different size screen. It even supports 'themes'; that's when you want to change the look of the presentation with different colors and such. They put in animation for popups and sliders, so I don't have to learn about that, I can just use them.

Sure, it may not keep up with the latest trends since it hasn't had a substantive change in quite a while, but that only means I don't have to keep changing things because it did. No, it doesn't support older browsers, but I don't use them; I keep my browser up to date. I can also copy the libraries to my Pi and keep what I have for a very long time.

It took me a couple of weeks to put the code together, and it looks like a whole lot of stuff already out there on the web, but it works with the way I do things. I still have my cool gauges to show various readings, and the charts that I use to tell me when something is starting to have problems. I managed to leverage a whole lot of the code I used for the original web interface to save me time learning how to do that all over again. So, I have my gauges, charts and control buttons. I even messed around a bit with a custom display that looks pretty cool for the thermostats. Let me take you on a quick tour; first a picture of what the main screen looks like on a laptop:


Boring right? I only show two gauges on the initial screen because that's really all I'm interested in most of the time. The temperature and the power usage right this minute. The other items are buttons to separate pages in the application that I can go to for more information. Looking at the thermostats:


Yes, they have color and are too far apart on the screen. The resolution of the laptop does this. I could easily expand them to be larger, but this works for now and adjusts fine on the phone or tablet. This is what it looks like on the phone in portrait:


This is the way it would look on a tablet mounted to the wall where I walk all the time. Tells me what I want to know in a glance. Similarly, in landscape:


I personally think this looks nice as a two thermostat control panel. Tells me everything I want to know at a glance and a little interaction can make changes. More about the way to change things in a moment, first, look at the presentation of the first page in landscape:


The two gauges are clear and easy to read. Notice I finally got tired of the brass gauges and went to pastels and chrome. I'll get tired of this too, but for now it suits me. In portrait:


The two gauges are still clear, but now they're vertical; I got this display for free with the JQuery Mobile library. I can scroll down the screen to see the other options by just brushing the screen:


In case you're wondering, the footer is a marque that slides from the left across the screen; that's why it looks different on each picture. Now, back to the thermostats and how to change them. When I tap on the little gear on the bottom right of the thermostat it slides in a popup that holds the controls:


Here I can change whatever I need to and then tap the 'Set' button to send the changes off to the thermostat. Slick huh? Yes, the house runs warm during the day; I live in the desert remember? Also, I don't like high electricity bills.

Yes, I have the charts on there. For example, the house power and outside temperature are displayed by the 'House Chart' button:


This is landscape mode, the portrait mode looks too crowded since all the data is compressed to fit on the screen. This is touch zoomable, and I can zoom in on any part of it. This comes free with HighCharts, the charting tool I've used for the last couple of years.

I do the same thing for the fridge, freezer and freezer in the garage:


That's the house freezer with its stats for the last 24 hours, and when I tap on it:


The chart for its activity during the last 24 hours slides in and displays. I even enabled the points on the chart for examination. I can keep really good tabs on the appliances I've implemented so far. I really want to include other things over time in a similar fashion.

The pool looks like this:


This tells me the pool motor is running on high, the waterfall is on, the light is on and the fountain is on as well. Yes, I turned them all on for this screenshot. The motor is black, green, red for off, low, high. I tried to animate it, but is looked silly. Each item is a button that can control that accessory, for example:


Tapping on the 'Off' button will actually turn the fountain off. There are other things I want to add like the temperature of the water and actual speed of the motor, but I haven't gotten to them yet. 

No, I didn't forget the weather station:



Each of the parameters I measure has its own gauge and shows the last reading I took. In portrait mode they are a series of gauges arranged vertically. When I have enough data to chart it accumulated, I'll add graphs for them. I'm not sure what's the right way to do that yet. Do I need 24 hours or the last week? I'll get to that someday.

The Statistics button on the bottom shows this:


I'm going to add stuff like rainfall this year, last rainfall, highest historic wind speed; you know, the stuff that impresses people that don't realize how easy a database query is to code. (Don't look at the low temp above, that's not a bug, it's a feature)

The last thing I'll bother you with is the control page for the lights. I currently only control four of them here. Here's the display as it normally appears:


All the lights are off. I do this control differently, each light is a button that can be toggled, so tapping the patio light will turn it on. So if I tap a couple of them the page will look like this:


Kind of adds a little color to the display. These are controlled by wemo light switches that I described breaking into through several posts.

Like I've said annoyingly, this is one source for all devices that I may want to use to talk to it. I kind of hated giving up on the other attempts, especially the Android native code, but it's already out of date and several versions behind in the Android system it runs on. It's unlikely the web will change that fast and the latest thrust is to use web applications anyway.

Those of you clever enough to figure out the URL are welcome to take a look, but remember, be kind; it's running on a little Raspberry Pi. You can't actually control anything unless you figure out the super-secret high security password, and if you do, I'll just change it.

Now it's time to see about exchanging that phone ...

13 comments:

  1. Great write up, tons of ideas (as usual :) )
    Thanks for your blog!

    ReplyDelete
    Replies
    1. You're welcome. Drop by anytime.

      Delete
    2. Great work!
      I have also a very elaborate website under jquery mobile and highcharts to view all the status of my home. I have one question for you, how do you make your website public but actions can only be controlled by you? The super secret password you talk about.
      Thank you and keep the good work

      Delete
    3. On the Pi, I have the password in a JSON file in a directory that can't be read over the web. Then I have php code that can't be seen as source from the outside that reads the file and compares the password with a parameter sent when I send a command. I type the password into the URL when I run it from outside the house. Inside the house, I check the subnet to see that it's a local address and don't bother with the password.

      It's not the safest approach, but there's no money involved, so it's good enough for now. I just (actually tonight) learned how to enable SSL on the Pi, so that will happen soon. The next thing is to munch the password in transmission so it can't be read off the URL over my shoulder.

      That should be good enough to protect it from someone trying to turn the lights on and off annoyingly.

      Delete
    4. Thank you for your reply. Its sounds like an OK method . I will continue to investigate what would be the best and easiest option. Right now I am only reading data but soon implementing some control also which will need security. My website is in a Synology NAS where the raspberry saves everything into mysql. If anyone has any suggestion I welcome it.
      Thanks!

      Delete
  2. Have you seen the project called Home Assistant (http://home-assistant.io)? I had a similar system to yours with JQuery Mobile, javascript Websockets and python web servers and I replaced it with Home Assistant. It serves as a decent UI and allows me more time to focus on the hardware units and firmware. It also got me to convert everything from a custom protocol/gateway (synapse wireless RPC) into MQTT. All my new devices have been Particle Photon wifi modules, which support MQTT natively.

    ReplyDelete
    Replies
    1. Yes, I took a look at it way back; and it has changed A LOT. I just took another look at it and it is pretty compelling in its current form. I dug into how they work with the Wemo switches and was disappointed that they use a link into the Belkin control stuff, but then again, they didn't want to break into the protocol and guts of the devices the way I did.

      I'm going to prowl around in there a lot more, it may be that I can implement my own methods for discovery and implementation to their system for the devices I have.

      The particle boards have been pointed out to me several times, but I hate wifi. My internet is so screwed up out here that I can't rely on it. At some point I may get reliable service, but it may be a while. Currently I have my house isolated from the internet with a router and a couple of UPS's so I can sill have the in-house network live when trouble happens.

      Thanks a lot for the pointer; I WILL be prowling around.

      Delete
    2. I just spent some time looking at the photon p0 again. This time it really impressed me that the thing can be loaded over the air without having to deal with a cloud service. I get the impression from the documentation that I can get all the stuff needed to program it over the air on my laptop and not rely on a cloud service at all.

      The question I have for you is: Is this really true?

      Delete
    3. Yes, you can program them locally. You will need to download the build environment and get it running locally. I'm not sure how hard that is. Last time I tried, the instructions were not clear and I agve up. I still use the cloud for compiling and flashing. I edit files locally and command like upload them to compile and flash.

      However, the MQTT libraries they provide connect directly to your local broker: there is no cloud involved. Once you get your devices programmed, all traffic is LAN over WiFi. You can even tell the firmware to not try to connect to the cloud upon bootup. I too am no longer at the mercy of a cloud service to be up for my home automation to work.

      The exeception to the cloud requirement is I was able to connect Home Assistant to my Amazon Echo device. I can now use Alexa to control my lights. I have found speaking commands compelling instead of pulling my phone out of my pocket to control the lights. Speaking allows you to be doing something else while you are messing with the lights.

      Delete
  3. You seem to like to build things yourself and have very particular requirements for things. I am close to the same. But I have been using HomeAssistant for months now, and I must say it is great. Also the community is glorious. Might just on the off chance check it out.

    https://home-assistant.io/

    ReplyDelete
    Replies
    1. I guess I should actually ctrl+f for "assistant" instead of "homeassistant" and I would have seen the comment right above me. I guess My comment is just me seconding homeassistant. Also if you haven't seen these awesome chips. Google for 'esp8266'. $3 dollar chips that have built in wifi and can be over the air updated. Down side for you may be the wifi, but I would still say worth a look. Mostly due to how cheap they are. Certainly check out http://www.wemos.cc/ . Also check out the library => https://github.com/marvinroger/homie-esp8266

      Delete
  4. I bought into the Onion Omega 2 kickstarter to try those out. From their page:

    "It is a Linux computer designed specifically for building connected hardware applications. It combines the tiny form factor and power-efficiency of the Arduino, with the power and flexibilities of the Raspberry Pi. With the Omega2, we want to lower the barrier of entry, and allow everyone to take the leap into hardware development."

    https://www.indiegogo.com/projects/omega2-5-linux-computer-with-wi-fi-made-for-iot#/

    It will probably take them a while to reach the maturity of the photon boards, though. Their software library is fairly robust these days.

    ReplyDelete
    Replies
    1. I understand the love affair with wifi ... but it just isn't reliable for me. Half the time is wasted retrying some connection or other. Plus, and here's the big one, what if you want to go through three walls or 150 feet? Wifi just won't do it.

      Then you have to think about another access point, spend a bunch of money on network extenders, something like that.

      My little XBees reliably send hundreds of feet outside and through at least two walls inside. If I have trouble with a particular location, just stuff another one in the middle as a relay. I've successfully talked from my neighbors house to mine. That's 400 feet through horse tack rooms, corrals, fences walls and such. Yep, I put a relay XBee in the middle to do it, but it was really, really easy.

      Plus, once you get to know them, they work first try every time.

      Delete