Sunday, April 12, 2015

Getting Back To The Weather Station

I know I've worn out my welcome about the AcuRite 5n1weather station, sorry. This time it's a bit different and I'm really, really open to input and ideas. I decided that the console that came with the weather sensor is too strange and quirky to deal with long term and set up a Raspberry Pi to act as my weather station. I used a radio and directly receive the transmissions from the weatherhead <link> instead of the USB interface I worked on forever <link>

I have a lot of work to go on it, but I'll update it over time as I figure out what I actually want. To date, I set up a little SQLite3 database and record each of the readings there on a periodic basis. Currently, I update each record either as it comes in, or every minute. The reason for the difference is that my barometer and outside temperature sensor is on my XBee network and handled by a different machine, so I only interrogate it on a minute basis.

Yes, the little Pi weather station is a separate machine and I have it talk to the Pi that controls my house to get the two readings.

Actually, let me elaborate on this a bit. Remember way back when I posted about bringing up an HTML interface for the various processes I run using CherryPy <link>? That little effort worked incredibly well. For most processes I have at least two interfaces, one that I can read and one that is formatted in JSON that a machine can read. So, to check on a particular process, I just type the IP address and a port number into the browser and the process responds so I can see whatever I set up. This interaction doesn't care what machine it's on or where the request came from. Using this I have the weather software on one Pi, then get the readings for barometric pressure and outdoor temperature from another one. It's so cool.

I may expand on this idea and have several Pi's running doing different things for me. I could put the XBee network on a machine by itself and have other things talk to it to get the data. Sure it raises the risk of failure because it increases the number of machines that can fail, but It also means I have all the power of a Pi to do a task and not have to worry about things bogging it down. I may move all the upload processes to a single machine so that stuff is separate as well.

But enough of that, I'm here to talk about the weather station. I keep data on a periodic basis for: Rain counter, Outside Temperature on the fence, Outside Temperature at the weatherhead, Barometric Pressure, Wind Speed, Wind Direction, and Humidity. I also keep a daily record, written at midnight that hold the Barometric pressure at that moment, the high and low temperature for the day, and the rain counter at that moment. I create this record once a day, so I can use it as a daily data point on a graph.

From this I can calculate the daily rainfall, and when I have enough data, the weekly, monthly, etc for my location. I have enough data to graph the temperature on a daily basis, and even use the midnight record to go longer term. That stuff, of course, will depend on gathering the data over time. Or, I could download the data from Xively and fill it in for the past if it becomes relevant.

This looks like it will eat up about a megabyte a week in space, and I have roughly 4 Gig available on the Pi, so I have some time to get an NAS online to hold the data long term.

But, as usual, I have no idea what the heck I'm doing. I don't have a clue what I may want next month, or what is the best way to gather the stuff that I don't know about yet. The station is running right now, but it isn't hooked into the web yet so I can't show it off. I'll get to that in a week or so.

I looked into loading the stuff up to Weather Underground, but that didn't excite me very much. I've been fighting cloud providers for a few years now and I'm not impressed too much anymore. With the terabytes of NAS storage available these days and the ease of bringing up a significant little computer, who needs the headaches of specialized protocols and changing terms of service?

So, you folk that have weather stations and such, what am I missing?

Oh, the code isn't on GitHub yet, I'm not sure how I want to arrange it there. I may set up another repository for this instead of adding it to what's already there. I need to think about this a while. But, if anyone wants to take a look, let me know and I'll move more quickly and make it available.

The next post on this project is here <link>.

4 comments:

  1. I didn't know what I wanted when I began storing data from my weather station a dozen years ago. Storage not being a problem, I stored all sensors data in a MySQL database every 5 minutes. I also stored them in a RRDTOOL database for graphing.

    As I started to know what was useful, I started rolling up the MySQL data into daily, weekly, monthly tables. Things like daily temperature high, low, and time it occurred, and averages. This was to reduce the time to create the web page. Then I could truncate the main database, but never did.

    Graphing 24 hour temperatures and such gets less interesting in time. That is partly why I used RRDTool, a round robin database. I can still graph a year, but beyond, I don't care.

    Now for example, I can easily see how many days that we had freezing and below in April for the past 8 years. The number of years I have lived at my current address. It gives me an idea when I can plant my garden.

    Chris

    ReplyDelete
    Replies
    1. I was hoping you would chime in Chris. I've seen Chris' weather display, and frankly, it's really great.

      The idea of RRD is appealing, and I may well do that too. I've noticed that monitoring my power gets boring for long stretches, but then I get an idea and pull up certain day's usage to see what happened. A good example is what happens to the power I use on a rainy day, the hottest day this month, that kind of thing.

      You've given me some great ideas, thank you.

      Delete
    2. Thanks Dave, I have to confess that I didn't get it to where it is at without a lot of help from others. A weather station called Temp'O Matic written by Timo 'Boozeman' Viinanen really got me pointed in the right direction. Dean-O's Toy Box (Google it) gave me great ideas for dynamic updates.

      There is one thing I wish I would have done but never could decide on how. That is recording the sky conditions. We have had way more sunny days this year here in Tigard Oregon than what I remember. It is really remarkable. I wish I had collected that information just to compare from previous years.

      I wrote code to parse the forecast from NOAA could have used that, but I just didn't. I guess I should do it so next year I can compare, if we go back to rainy almost every day.

      Delete
    3. Ha, I could forecast sunny and warm every day and be right 335 days a year and have a better record than the weather services. The weathermen here complain that the weather is boring and not really worth wasting time on every evening.

      Then, we get three inches in 20 minutes and it floods out entire towns.

      Sigh...

      Delete