Exporting Data to CSV [implemented!]

[ekoelle]

This may be a pretty basic question but is there a way to export the historical data from my sensors to a CSV file?

[Felix]

UPDATE: This CSV EXPORT feature is now implemented, see below for video demo.

It's a really good question actually.
The answer right now is ... not really 
The data is stored in binary format. I would need to write a program that exports it according to given date parameters (start end).

Maybe a utility page that allows entering 2 dates and starts a CSV file download?
It could also display the START and END of the stored data, and # of points.

[ekoelle]

Maybe a utility page that allows entering 2 dates and starts a CSV file download?
It could also display the START and END of the stored data, and # of points.

That would be ideal. I'm looking to implement a temp/humidity tracking system in our warehouses at work but we would need to be able to export the data periodically for generating reports. I'd like to avoid having to tap into an online service as i'd like to keep the system contained within our network and not reliant upon someone else's servers. Is all of the data contained in the gateway.db file?

[Felix]

I'd like to avoid having to tap into an online service as i'd like to keep the system contained within our network and not reliant upon someone else's servers. Is all of the data contained in the gateway.db file?

That's exactly why I created this gateway software

Is all of the data contained in the gateway.db file?

No, the historical logged data is kept in the dedicated log subdirectory, as binary files for each node-metric.

[Felix]

Happy to say this CSV export feature is now implemented, demo & other features can be seen in this overview:

Clicking the new Export-raw-data button creates a CSV file that you can download and process as you'd like (timestamps in milliseconds since unix epoch):



The formula used in the video to re-format the timestamp is this: =A2/(1000*60*60*24)+"1/1/1970"+(-5/24)

[Nutz]

Thanx Felix, this is great. I tested it on my installation and works like a charm. Was busy implementing something similar but you beat me to it.

[Seedser]

Felix,
Can I export all saved data? I assume data amount is only limited by RPI SD or connected storage device size?
I'm looking to record temperature over the year.
I would like to save at 15 minute intervals.
Data points per sensor per year = 35,040.

Mike

[Felix]

This function will export the raw data in the zoomed range. So zoom out to a year, then export to CSV, what you have in the binary DB log is what you get, there is no 15minute interval functionality.
If you want 15 minutes data, then send it in/record it at 15 minute intervals.

[EnMon]

Happy to say this CSV export feature is now implemented, demo & other features can be seen in this overview:

Clicking the new Export-raw-data button creates a CSV file that you can download and process as you'd like (timestamps in milliseconds since unix epoch):



The formula used in the video to re-format the timestamp is this: =A2/(1000*60*60*24)+"1/1/1970"+(-5/24)

Hello Felix,

Is there a possibility I could insert this time-stamp formula into my gateway.js code, so I download the excel sheet with the actual date and time as opposed to the current way of manually inserting the formula every time I download the CSV file?

If yes, could you point me in the right direction please?