Over and over again I've ranted about how I hate cloud storage of data, or reliance on cloud services for control around the house, and again I was justified. Earlier this month Xively (formerly Cosm (formally Pachube)) turned off free access. Never mind that years ago Pachube promised access forever for its early users and promised again when they became Cosm. Xively didn't honor the promise.
So much for promises. Remember the wonderful free service DynDNS that had its software installed on new routers? That went away a while ago and I had to change away from that service, now I get to stop the update process for Xively and pull the data from my local database instead.
At least I'm ready to do the switch, and I wish I had bothered to do it a year ago, but I was lazy. Teach me...
They were pretty nasty about it too. They sent emails to some fraction of their free users, and ignored the rest. Even the users that got the mail only got about two weeks notice and were unable to suck the data off the cloud service in time. The others (like me) found out from friends or just noticed one day that the service (and all their accumulated data) had disappeared from the site.
They couldn't even log in, and mails seemed to disappear into the same black hole as their data.
I don't recommend any product from LogMeIn. For software to avoid, see the Wiki page on the company and avoid all their offerings. The nasty way they did this should be a nice hint for folk.
I'm not going to recommend a replacement because this situation is only going to get worse over time. Hold your own data at someplace that belongs to you, not some profit driven jerks.
Now, where was that charting software I like.
Yup. I can't agree with you more. Just wait. The same thing will happen with Thingiverse and the three D print files they store.
ReplyDeleteI have been using homeassistant for a while. That basically handles a lot of my home automation stuff. It also has a bunch of plugins, one being an influxDB plugin. With that I also am running a graphana container which is pretty configurable on how to graph data from my influxDB. I would try looking into graphana if you have the time. I own all my data. :)
ReplyDeleteOK, fine. I'll take another look at homeassistant. Back when I started this process there just wasn't any good software available to do the kinds of things I've done around the house.
DeleteBut, it's been years and years of 'roll-my-own' and I should go look around again.
a +1 for homeassistant from me as well. Influxdb + Grafana is incredibly powerful.
DeleteFor a less frills versio...Sparkfun created did "phant" and it can be self hosted. https://github.com/sparkfun/phant . I wrapped it in a docker container for easier consumption here: https://hub.docker.com/r/jturpin/phant/
OK, I understand the premise of Docker and have tried several times to come up with a reason to deal with it, but haven't gotten there yet.
DeleteDocker just seems like a ton of work to get something running on something like a Pi when you can just run whatever it is you're working on. I hate it when people come out with a really cool something or other that has to have docker before you can try it out.
Actually I'm taking a really close look at homeassistant to see what all the hype is really about. And so far, ...
Ah heck, I'm just going to post about this endeavor so folk can guide me some. I may even join the forum to help me understand the attraction.
There's definitely a learning curve to docker. Once you get the hang of it it's pretty easy and really does simplify things. I was in the same shoes as you and then started using docker in my day job and it really fixed a lot of things. Specifically, "It worked on my box."
DeleteAnd people keep thinking that I'm strange when I say that I avoid Cloud services... including paid ones!
ReplyDeleteI moved mine to ThingSpeak but I don't really care about the long term data. I may look at it in a chart now and then, but that's about it. Someday I may do the same, but I'll have to figure out how to only keep a few months or so worth of data. Unless I find a need for the long term data, that's likely all I'll worry about. Until then, I'll be watching to see how your adventures work out (as always)...
ReplyDeleteIf you're not going to keep it past a couple of weeks, there's a ton of relatively easy solutions. For long term storage (years and years) gotta use a database of some kind.
DeleteEven though I have years of accumulated data, I haven't actually used it for anything. I may have to rethink this.
That's where I stand. I may look back at something over a few weeks, but rarely do I care about it much past that. Especially to the extent that I'm sampling the data during the day. every 15 seconds in some cases adds up to a LOT of data. Granted, I can look and see what the temperature of a given item has done over time, but do I really care about what it was this time last year? I'm not sure of what you're calling easy solutions, but for the moment I'm simply using someone elses database services. :-)
DeleteUsing someone else's services is the easy way ! No maintenance, minimal down time, etc. We just have to be ready for them to disappear...
DeleteFor me, there is value of tracking over long periods of time and this is where a time series DB like influx can really shine. You can configure it to down sample readings over time.
ReplyDeleteWhy might you want longer historical data? Think about long lasting upgrades to a house. It sure would be nice to determine how much of an effect installing some new windows or adding insulation had compared to the same time from a year or two before.
I have a number of years of back data and have seldom used it. I keep it for all the reasons you mentioned.
DeleteI agree, but I keep records of utility bills and such and it wouldn't be difficult to pare down readings for longer term use. Like I said, I'm sampling some things at 15 second intervals. I get the reasons for long term data, just haven't used any of it and not sure it's worth the storage requirements and such. Besides, unless you're using RAID, you could still lose it all in a fraction of a second...
DeleteMichael, you're exactly right. Although I am using RAID, I could still lose it all in very little time. There's just something comforting about having all that stuff stored up in my attic with the Christmas decorations.
DeleteI've had so many electronic or programming failures around the house that I completely understand the vulnerability. Actually, every device I came up with was in answer to some problem I had.
That reminds me, I need to fix the septic tank level sensor ... the rats ate the wiring.
GroveStreams seems pretty reliable and helpful, though it's free only for small quantities of data.
ReplyDeleteWill