Thermostats, Location, Location, Location

When I was researching supercooling as a way of limiting my power bill I kept getting readings and such that just didn't make sense. One side of my house was comfortable, the other ... not so much. Sure it was the sunny side this time of year, but it has really good windows and good curtains. The only two major problem areas were a couple of windows that faced directly into the after noon sun.

I fixed those years ago with reflective insulation that I put in for the summer and (sometimes) take out for the winter. It's easy to work with and will last forever if I just give them a little attention every once in a while. I wandered around the room and noticed that the interior walls were about 2 degrees warmer than the rest of the house and there was heat getting in through the foundation.

The sun bearing down on the exposed slab foundation would heat up the concrete slab and transmit heat inside the house. That's something that might go unnoticed if I had carpeting, but the tile felt warmer as you approached the wall.

So, basically, it was like any other house on the sunny side. Why was my AC running all the time and having little impact on that room?

A little background: I have a house shaped like an "L" and two AC units. The one on the north is a 3 ton and the one on the south is a 5 ton. Both of them going could cool the house, but when they cycle, it doesn't seem to handle it. I had two thermostats that were located by the AC 'experts' that did my house and were in the top middle of the price range. I replaced them with highly programmable thermostats that were in the high range of prices. These wouldn't do the job of handling peak periods well, so I got a high priced "learning' version. It was crap.

The learning thermostats only learn what the programmer wants them to learn, and it was great at anticipating my needs. It decided I liked the AC on at 07:00 and off at night ... exactly the opposite of what I wanted. They tried to predict what temperature I wanted the house at and must have been born in Nova Scotia because they wanted to keep it at around 65F.

I sent those things back before the 30 day refund period was up. What a waste.

Then, I built my own <link>; I could control them any way I wanted to. Over the years I tried recirculating the air around the house with them, experimented with differing hysteresis curves, used various smoothing algorithms on the temperature readings, etc. Basically, I tried everything I could think of to get the environment the way I wanted it.

Nothing worked very well. I would still wake up with the bedroom running around 85F at night.

But, during my experimenting with supercooling the house (look at the posts on supercooling by using the contents search on the top right of the page) I noticed a whole bunch of things that really shouldn't be happening. First the south AC unit was short cycling. Short cycling is where the AC comes on and then shuts off in seconds or minutes. This is NOT a good thing.

Cycling like this can radically shorten the lifetime of any motor driven device. It causes undue wear on the bearings to be jammed up by the quick start of the motors. It can cause fan blades to slip and that scores the shaft. The air start up can loosen vents and cause rattles. Basically all the things that destroy devices like this. This behavior can also cause a room to be too cold and then too hot within minutes.

The very most expensive AC units start slow and ramp up to speed, they will run slowly to keep the air moving, they even open and close vents based on the need for cooling in a particular area. But, I'm not Bill Gates; these things are way out of my price range.

So, I started looking for the cause of this situation on my south AC. TLDR (too long didn't read) the thermostat was in the wrong place. The thermostat was in a hallway like most of the thermostats in the world close to an overhead air return for the AC. Over half the air for the house went through that return. The temperature would drop rapidly in that location when the AC turned on and cause shut off in a matter of a couple of minutes. Then it would rise rapidly back up because it was only the breeze that cooled it off and repeat the cycle.

Another factor was that I run in and out of the garage a lot. Each time warm air would hit the thermostat and cause the same problem. Add to that the garage is on the other side of the wall and it doesn't have AC. When it heated up, the wall was warmer than the air in the house; cycle started all over again. Obviously, the thermostat had to be moved to get better performance.

Based on this research, I modified the house code to save the activity of the two AC units and looked at them for a day to see how bad it was. It was pretty bad. Here's a chart of my findings on the first day for the south AC unit:

The tall vertical stripes are when the compressor was running, and the jagged line down the middle was the temperature the thermostat was reading. Each shaded long vertical strip is the compressor chewing up power and wearing itself out. Here's a closeup of a couple of them from the middle of the chart:

The stair steps in the temperature is due to the granularity of my recording the data. I only mess with one degree changes, so it comes out like this. So, you can easily tell that the hysteresis is three degrees and it takes very little time to overcome that with the AC unit essentially blowing down the hallway from all the rooms. Similarly, the temperature will shoot back up in a similar fashion because that short a cooling interval doesn't really cool anything except the air the it is blowing around. The gap in the middle is because the cumulative impact of a series of cooling cycles finally does cool things down a little.

When I looked at the North thermostat, it was a different set of environmental factors that added up to a different problem. This thermostat was in a short hallway outside the master bedroom (my room darn it) while the AC unit was venting into that room. There were also a couple of vents that dumped air into the rest of the house to aid there as well. When I looked at it, it looked like this:

Same things on the graph, compressor run period and temperature recorded by the thermostat. Notice the compressor runs a lot even though the temperature isn't changing? Part of that problem is that the thermostat isn't really looking at the AC activity, it's looking at a completely different area. The rest of the problem is that the two vents into the rest of the house are letting almost a great a volume of air get out there as it allows inside MY ROOM.

Fine ! I gotta move the thermostats, but where the heck am I going to put them? Also, it's a real pain to move a thermostat. I don't have a real attic, it's an area that is totally filled with insulation about two feet tall. I really, really don't want to go up there --- ever. I could carve out the wall and move the thermostat and then patch up the wall. That's equally unlikely to happen. Instead I experimented with really long hysteresis values. I was up to 10 degrees at one point. That solution was not satisfactory. I also tried running the fan longer to make sure the air got moved. That didn't work well either; it actually made the problem worse.

Then, I thought about it a bit. I had these temperature sensors <link> all over the house, why couldn't I just use them instead of the temperature sensor inside the thermostat? There wasn't a single reason why not except I would have to change some code in a few places. I took all the code out of the south thermostat related to reading temperature and reorganized it a bit. Now, it would receive the temperature forwarded by the house controller from any of the temperature sensors around the house.  I chose a room where the sensor was on an inside wall away from any breeze from the AC directly. The sensor gets air circulation, it just wasn't directly in the cool air path.

Now, I was ready to test the newly modified south thermostat. The one that was banging the compressor to death with short cycles. Let's see what this accomplished:

Now, that's more like it. Granted, it was a stormy day and not our usual 100+ temperatures, but look, it wasn't short cycling any more. There are long periods of AC activity and the temperature rises slowly instead of a few seconds at a time. This may look like I'm actually using more power, but I'm not. It's actually about the same because the gaps are longer as well. the big thing was that I wasn't tearing the guts out of my AC.

I did the same thing with similar results to the North AC, except I also closed off the vents to the rest of the house. Now my bedroom cools down pretty quickly and stays that way for a while.

Notice that it isn't on all the stinking time blowing air where it isn't needed? The big gap is the 'demand' period where the AC isn't allowed to run 15:00 to 20:00. After the down time, it kicked on and recovered a little after midnight.

The beauty of this is that I can build a couple of temperature sensors specifically for the AC units and put them anywhere I want. If I want priority in the bedroom, put the sensor by the bed. If I want it for the shower, put it just outside; you get the idea. Basically, this is a priority system now that allows me to carry the priority anywhere I want.

<rant>
I hereby copyright this idea !! Anyone is welcome to do anything similar; heck steal my code if you want, but you commercial shark guys out there better get permission to use this. It's real annoying when you find your code, stolen right off the site and only perfunctorily changed in a book that sells for $35 a copy at a book store. I'm looking for new ideas and find my old ones on the shelf...sigh. I have found my code and ideas all over the web and almost every single time it's attributed; that makes me feel good. When someone is selling it, not so much. This stuff is supposed to be free dog gone it.
</rant>

Anyway, the jury is still out on this idea. I need to watch it over time and see if there are problems with the devices talking to each other, getting out of hand unexpectedly, actually working the same way over time. That kind of thing. One thing I did notice though is that a web only version where there is no control station nearby sucks. It's really nice to be able to lower or raise the temperature by pushing a button on the wall instead of starting up a browser and working through pages to get to it. Now, this could be a cheap tablet, and old phone, or something like that if you want to get fancy, but a nearby control is nice. I like buttons.

Also, it's a good thing to have a display that shows everything is working OK. I'm currently using the display on the thermostats, but it could be anything that you can see when wandering aimlessly around the house looking for the car keys. This also helps with the, "It's too hot." comments. You can point to the thermostat and show them that it's only 80 and not hot at all. Folk visiting from the East do this all the time.

Another thing I learned is that AC 'experts' are not to be trusted. These things were professionally installed and came highly recommended. They put one thermostat outside the room it was needed in and the other in a breezy hallway next to the air intake and a garage door. In retrospect, I may be to blame somewhat for not asking enough questions, but these were experts hired for their knowledge and experience. I don't have a suggestion on how to handle this problem except to get two or three experts to present proposals and pick their brains. If you are Bill Gates and can afford to do that kind of thing. I even blogged about the unreliability of 'experts' a couple of times; my experiences are not good with them. Of course, now that I can put the temperature sensor anywhere I want, it doesn't matter as much.

See what happens when you start automating your house? Ideas may come slowly, but actually doing them is entirely possible.

Oh, this is another post that will attract attention from the spammers. If you post advertising in a comment, it will be quietly deleted. Usually the same day. So, don't bother advertising your AC repair company here.

Local Power Politics: APS, AZCC and Things to Think About...Continued

No, this isn't turning into a political blog, although, it would be fun, but this relates directly to my continuing annoyance with my power company Arizona Public Service. I have a few previous posts about how poorly (in my opinion) it's run and how they are using the power of the dollars we pay in rate fees to control local politics and raise rates. In the last post about this <link> I specifically mentioned a particular rate increase that is being contested right now. I'll leave the long description of that item to the reader; simply check out the link I just left you.

What I want to recap here is one particular item that is turning out to be fun. Briefly, an APS customer, Stacey Champion, is petitioning to have the latest rate increase revisited. She contends, and I agree with her, that APS dealt improperly in their presentation of data and impact of the rate increase. A lot of people are behind her in this effort, and two individuals have joined directly in the proceedings to help out and also get some of their own particular opinions aired.

Naturally, this is becoming an interesting problem for the members of our regulatory commission, Arizona Corporation Commission. Basically, the AZCC is the equivalent of a Public Utility Commission in other states. They have commissioners that vote to control rate increases and act in the public's behalf. Many contend that APS owns some of the commissioners, and it's hard to deny when millions of dollars have been spent by APS on political campaigns locally.

So, this particular 'Docket' is a hot button right now since some of the commissioners are up for reelection in November.

Now, to the fun part. One of the commissioners Tom Forese sent a note to the people handling the Docket asking for the inclusion of 'Staff':

Dear Commissioners and Interested Parties,

I would like to request that Staff participate in the matter of Stacey Champion, et al vs. Arizona Public Service. Staff was a party to the rate case, and Staffs [sic] participation will assist me in making my final decision. 

I would like Staff to respond by close of business June 29, 2018.

<link>

My first question was who is this 'Staff' person? Is there a Joe Staff that works there or what? Well, it turns out that 'Staff' is the commission's legal staff. Forese is actually trying to inject more lawyers into the mess. These lawyers are the same ones that worked on the previous Docket that is being contested right now.

'Staff' replied with another note:

Because this proceeding has been underway since mid-February 2018, Staff respectfully requests that a procedural conference be scheduled at the earliest possible date to discuss Staffs participation in this case and to consider this proposed procedural schedule to accommodate Staffs involvement 

<link>

I only quoted the last paragraph because the rest of it is legalese, and you can see it all by following the link above. Below this is a strawman schedule that is proposed so the lawyers can get all their poop in one sock and not look like a bunch of dweebs.

Notice that the lawyers ask for a conference. In this case it means that a judge presides, and all the folk directly involved have to attend and have their lawyers there. It's really a hearing; you know how lawyers love a hearing. The administrative judge involved, Jane L. Rodda called for a conference (hearing) to be held on July 11th to discuss this and other matters.

OK, this sounds a little weird on the surface. The commissioners have not attended any of the hearings so far, and have had little obvious involvement. This has been between normal folk and APS's plethora of lawyers. Now, one of the commissioners, actually the chairman of the commission, wants to inject lawyers to tell him what is going on. Initially, I wondered, "Why hasn't he been following this already?"

Turns out that I wasn't the only one wondering this. Warren Woodward, one of the normal folk that is also directly involved in this with Stacey Champion filed a letter contesting 'Staff' involvement. This is the really fun part and I'm going to include most of it below:

Warren Woodward, Intervenor in this proceeding, strongly objects to the suggestion put forth by Staff on June 29, 2018, that a procedural conference be scheduled in this case to discuss Staffs participation in this case. 

1) This matter can be handled on paper. There is no need to waste the Parties' time at a conference.  

2) The time to apply for intervention passed on May ll, 2018. Staff missed the boat.  

3) By virtue of signing the APS rate case Settlement Agreement, Staff is biased in favor of the rate hike established in ACC Decision 76295, and Staff cannot be expected 1 to be impartial in this case. To wit, Section 40.6 of the Settlement Agreement:  

• The Signing Parties shall make reasonable and good faith efforts necessary to obtain a Commission order approving this Agreement. The Signing Parties shall support and defend this Agreement before the Commission. Subject to subsection 40.5, if the Commission adopts an order approving all material terms of the Agreement, the Signing Parties will support and defend the Commission's order before any court or regulatory agency in which it may be at issue. 

4) If commissioner Forese, who suggested Staffs involvement in order to assist his decision making, wants to better understand the issues in this case, then he can read the docket, including all the APS customer complaints, both in this docket and in docket E-01345A-16-0036. There is also nothing stopping commissioner Forese from sitting in at the hearing and paying attention, something he neglected to do during the APS rate case in which these unjust and unreasonable rates (that are the subject of this proceeding and for which Forese voted) were spawn.  

5) In light of all the foregoing, there is really nothing to discuss. Staffs request for a procedural conference, and Staffs intervention in this case, should be denied.

<link>

Is this AWESOME or what? Woodward stepped up and actually said what should have been said:

This herd of lawyers has not been paying attention to this because it seemingly wasn't important, therefore they want to extend the schedule of the hearings and actions. Probably past the election so it doesn't interfere with the sitting commissioners campaign for reelection.

That the herd also cannot be neutral, or even in favor of the public's interest because they signed off on the rate increase and promised not only to not object, but also to "support and defend" it. Including them will add lawyers that are required to support APS in this matter.

The Chairman of the Commission himself isn't tracking it and doesn't want to go to the effort of catching up so he wants the lawyers to do it for him.

I'm going to add something to this that wasn't directly mentioned. More lawyers means that the normal folk that are trying to get this reheard will have to spend more money answering the new lawyer's questions. More time will mean more expense and delays so this falls further behind in people's minds. We all know that more lawyers always cost more money, and APS doesn't care at all. They are guaranteed a reasonable profit by the regulating body regardless of how badly they run their business.

One other thing, how could a section like 40.5 quoted by Woodward above ever have been signed? The parties to this 'settlement' were the solar industry, other power companies and such. My Lord, it also included Walmart and Sun City for crying out loud. There's a paragraph in there that says they will defend it?

What?

And it's really there. I found it on page 31 about half way down the page <link>. If you want to see all the documents and interactions look here <link>, but it's really, really boring.

OMG!, do we have the best regulatory body that money can buy or what?

PZEM-016: Another Chinese Power Monitor

I really like the PZEM-004 that I picked up just to see what it could do <link>, in fact I built the monitor and control system for my water heater with it <link>. This thing has really taught me a lot about water heaters and how much money my solar water heater actually saves me.

That led me to look at other devices available from China that could actually help out around the house. Well, the same manufacturer makes a different model, the PZEM-016 that does much the same, but even better for my purposes. This one does the measurements for me, but also has an RS485 serial interface so I can watch more than one of them.

OF COURSE I took it apart:

It's built extremely similarly to the 004 model I already wrote about. The big difference, of course, is that this one doesn't have any display. That's OK, I'll take care of that part. But first I have to actually talk to the darn thing. I've already worked with RS485 using an arduino when I put together my pool controller <link>, so it isn't too strange, but it still intimidates me a bit.

I dug around in my boxes of left over pieces and found an adapter to go from TTL serial to RS485 and combined it with an arduino and started working on getting it going.

Naturally, it was a pain in the bottom to talk to the device. I emailed the manufacturer when I first ordered the devices (yes, I got five of them) for as much documentation as they could provide. They sent me a manual that was actually pretty easy to read and understand. In it they said that the device was Modbus compatible, and that really impressed me. If you look up Modbus, it is an industrial protocol for machines. It can control a large number of devices in an industrial setting and should have code that I can leverage to get this working.

Right ! Things never ever work out that easily. I did find protocol libraries that I could use on the arduino, but have you ever looked at Modbus? I thought the documents for ZigBee were obtuse, these are where ZigBee learned how to do it.

Frankly, I chucked the idea of using a Modbus library down the tubes pretty quickly in favor of a much simpler approach. When I looked at the messages that the PZEM 016 actually used, there were only a few of them and the responses were pretty much canned and easy to work with. I just sat down and put together a message to read the data from the device and sent it to see what happened.

No, it didn't work first try. No, it didn't answer on the second or third try either. One has to understand that if you don't get the message exactly right, you'll never get a response from the device. In my case, I was messing up the checksum. Fortunately, in the last couple of years there have been many sample checksum implementations and online calculators implemented. I tried a couple with my data and hard coded the actual message I needed to send, that actually got me a response.

Then I spend a couple of afternoons working the kinks out of getting the response and using the checksum to validate the message. Once I could send and receive a single message reliably, I was ready to start adding code. Naturally, it encountered problems. It seems that short messages would cause checksum problems ... sometimes. So much for the idea that a computer does the same thing each time. I worked at this for quite a while without resolution. Here's a couple of screenshots of the arduino serial interface. The first one is using a message that requires a short response and the second is a longer message. The interesting thing is the accumulators I stuck in the code to count the checksum errors.

The short response has 34 checksum errors out of 100 tries while the second longer response has only one out of 100. Same code and timing in both cases...sigh.

For the rest of my experiments I used messages that required a long response. Eventually, I implemented code to handle reading the values from the device, changing the address of the device, resetting the energy (kWh) accumulator on the device, etc. I actually had it working pretty well.

Then I outfoxed myself and decided to modify the code to handle more than one device on a single pair of wires. This was actually easier than I thought it would be. The idea is that each device on an RS485 line has a different address, and you address the one you want to control or receive values from. In theory I could have several of these being read by a single arduino and monitor a lot of things around the house.

But, that would mean unique addresses and unusual delays and strange things happening. Gritting my teeth to the point of pain, I dug into it.

One of the initial problems I ran into was not knowing what would be coming after I sent a request out on the line. Sure, it should be predictable, but it never works out that way. When one does a serial read, you can get back something that is expected and just follow the bytes until you reach the end. We've all seen this: a protocol has a leading byte to tell you the beginning of the response, then a length to tell you how many bytes are to follow. You simply get the length and then read until the rest of them come in.

Suppose that is the last byte you see though. Or suppose there's a burst of noise on the twisted pair and you get about a thousand more? Obviously you can't rely on a length in the incoming characters until you can verify the integrity of the message by reading the checksum way out there at the end of the message. Let's make this problem even nastier, RS485 lines can ring. That means that you can get strange interference on the line that will mess up any message that is running around on it. You have to allow for settling times and such after messages fly around.

The problems are not insurmountable though, industry uses these protocols and devices every day. If they can make them work, I can get them to work well enough for my place. And, I think I did. Here's the code I came up with to read a message coming in:

int getit(){
  memset(rxbuf, 0, sizeof(rxbuf));
  int i = 0;
  if (digitalRead(debugPin)==LOW)
    Serial.println(F("Data from port:"));
  unsigned long startTime = millis();
  unsigned long lastChar;
  boolean startchecking = false;

  while(millis() - startTime < readTimeout){
    if(pMon.available() > 0){
      rxbuf[i++]=pMon.read();
      if (digitalRead(debugPin) == LOW){
        print8Bits(rxbuf[i-1]);
      }
      delay(1);
      lastChar = millis();
      startchecking = true;
    }
    if (startchecking && millis() - lastChar > 4)
      break;
  }
  if(i == 0){
    noResponse++;
    if (digitalRead(debugPin == LOW))
      Serial.println(F("NONE"));
    return(0);
  }
  if (digitalRead(debugPin) == LOW)
    Serial.println();
  uint16_t calcCrc = makeCrc(rxbuf, i-2);
  uint16_t rxcrc = word(rxbuf[i-2], rxbuf[i-1]);
  
  if (rxcrc != calcCrc){
    Serial.println(F("Checksum error"));
    if (digitalRead(debugPin) == LOW){
      Serial.print(F("Calculated "));
      print8Bits(highByte(calcCrc));
      print8Bits(lowByte(calcCrc));
      Serial.println();
      Serial.print(F("Received   "));
      print8Bits(highByte(rxcrc));
      print8Bits(lowByte(rxcrc));
      Serial.println();
    }
    checkSumErrors++;
    return(0);
  }
  return(i);
}

What I do is set a one second timer around the entire message and when a single character comes in, I set a intercharacter timer of four milliseconds for the next character. This way the most I can wait for a message is a second and then if it just stops mid message, I only waste four milliseconds before I give up and try again from the beginning. This works really well to cut the necessary time to read a message down as well as notice a failure quickly. I was pretty proud of this piece of code until a little later.

When I tried to send messages quickly, there were problems. One response would pile up on top of another from a different device. This required a delay between devices so things could quiesce a bit. Long painful experience has shown me that setting delays in code is just programming around a problem rather than solving it, but sometimes you just have to wait for other devices to stabilize before moving on. This is one of those cases because the devices on the line don't send you a ready message.

One other thing you'll notice in the code above is that I found a new debugging tool, an input pin. I use pin 3 on the arduino as a digital input pin and check to see if it is grounded before putting debugging messages out. If it's running and I see something I don't understand, just ground pin 3 and the debugging messages come out to the screen. I really wish I had thought of this about eight years ago.

The other pin I use for a special purpose is pin 2. If it's grounded I go into a special piece of code that allows me to change the address of a device. All the devices come addressed as one initially and I have to change them to something else to actually use them. So, if I add a device to the line, and boot the arduino, the first thing it does is check for a device at address one, and when it finds one, it tells me to change the address and hangs up in a hard loop.

I plug in a wire to pin 2 and then boot the arduino again. It senses the pin and goes into special code to allow me to readdress the device. This is also a good time to recompile and add another device to the device table. Yes, I took the cheap way out. I add a device in the code by changing a number and entering the default values as well. It just wasn't worth the time to come up with a more elegant solution for something that will happen six or seven times ... ever.

Basically, I'm done with being able to control and read this device, but that is the beginning of a greater project I've been thinking about for a long time. I'm going to put several of these in an enclosure and measure the power usage of my major appliances. The 2 AC air handlers and the 2 AC compressors are big users of power and I want to track their operation. The stinking dryer that has cost me so much money because people keep using it is another one <link>. I messed up though and only ordered five of the devices. I need one more for the kitchen stove. It'll be on order tomorrow probably.

Before the more astute of my readers comment on how an appliance that uses both 110 and 220 like a dryer or kitchen stove can't be measured with a single current transformer because one leg is referenced to neutral, go look at this post from quite a while back where I found a way <link>. Yes, Dorothy, there is a way to do it.

Here's a little sample of two of the devices monitoring my light that has two bulbs in it. I used the exact same setup when I worked on the 004 version.

I have the debugging pin grounded (that is so cool) and I'm reading the light with one bulb turned on. Notice the difference in the 'Energy' value? One of them has been recording usage longer than the other. They both read basically the same thing for the other measurements because they are hooked to the same thing.

Adding another monitor to the stuff is simple, Wire it in, ground pin 2, reboot, change the address to 4, change a value in he code, recompile and away it goes. Since this is one of those devices that will just run for a long time without changes, that should be fine.

Here's a picture of the setup I used to get it going.

The board in the upper left is the RS485 converter and the CTs are off on the right hand side.

Next, mount them in something, wire them up and hook the CTs around the power lines in the mains panel. I will add an XBee to the arduino and send messages back to the house monitor just like I did for the water heater. I fully expect to add at least one solid state relay (SSR) to the project to make sure the dryer is under my complete control. No more running the dryer during peak period for me.

Have fun.