In part 2, I discussed logging the data from my Current Cost device into a MongoDB database using a small app running on my home server. It turns out I have made a glaring schoolboy error, which thankfully I have caught before it causes an actual problem. Having had some time to collect data and start analysing it, I've also found a couple of other problems as well. I thought it would be worthwhile to revisit these issues.

Timezones

Basically the main problem is a simple one. Every time an instant reading arrives from the device—roughly every 6 seconds—I am logging it into the database, appending a timestamp representing the current/date time. This date/time is taken from the computer rather than the device, which I believe is the source of the other problems I'll talk about below.

What I forgot to consider at the time is the timezone, and the fact it can change. Twice a year in fact. Come March, when the clocks go forward, I'll end up with a strange "hole" in the data whereby no readings were recorded. When the clocks go back again later in the year, the problem is even worse; I'd essentially end up with two sets of readings from the two actual hours interleaved when sorted by timestamp. Clearly that's not right at all, and graphing that would be utterly meaningless.

The solution to this is simple: always record the current UTC time for the timestamp. This way, they will always be sequential regardless of what we are doing in terms of winter/summertime. When displaying the data on a graph or whatever later, the times can just be converted back into the appropriate time zone to reflect when they actually happened in relation to the viewer's (well, my) current clock time.

Lesson learnt: logging time-sensitive data needs to have an always-consistent time attached to it

More Time-related Issues

The other issues I mentioned briefly above are also related to the time, but not the time zone. The device itself has a clock on it (no date), and it uses this clock to determine when each hour is finished so it can send the bi-hourly updates and daily updates.

Essentially the problem here is that the computer clock and the device clock could drift out of sync. I think I have compensated for this problem by checking the device time (which also comes with each update) to determine what it thinks the hour is. That way, I should be able to make sure I record each two-hour slot for the two hours the device thinks it is reporting. To be honest, I'm not 100% sure I've got this quite right, but I haven't had any duplicates accidentally recorded for any given two-hour period so I think it's ok. Probably good enough anyway.

The daily updates are another matter—I have no idea what the problem is right now, but I'm getting all sorts of weird stuff in there. That's a bit of a shame as that's probably one of the more useful figures I'm getting in terms of helping me set energy consumption reduction targets and so on. Since the data is historical in nature, I hope to have it sorted before I actually start losing data.

Lesson learnt: trying to synchronise things across two devices with a different idea of the date and/or time can be a bit confusing.

Do It Myself

I could sidestep this issue entirely by doing all the analysis of KWh on the server rather than using the device's figures. I'm not entirely sure whether the figures it calculates are using a simple algorithm (Watts × Time) or if there's a little more to it than that. If it is the former, I should be able to run over all the data myself and calculate the KWh numbers which ought to match the ones from the device. I plan to try this out at some point in the near future, as I'd prefer this implementation personally.

Lesson learnt: sometimes the more complex solution is the simplest. We will see.

What's Next

Hopefully I'll be writing up part 3 in the next week or two. This will cover the web front-end which I will have created for viewing the data. Needless to say, this is something which can be almost infinitely extended (factor in the current energy cost to see how much each day/month is costing), but the short term aim is to have something capable of giving me all the key figures in an attractive and non-taxing way. Expect screenshots.