I have a payload / data sets with about 130 character and I wonder what opportunities I have to transfer the data via RFM96 modules:
This is a demo data set and description:
Date/Time, Weight, Outside Temp, Outside Humid, Inside Temp, Inside Humid, H1 Temp, H2 Temp, H3 Temp, H4 Temp, H5 Temp, H6 Temp, B1 Temp, B2 Temp, B3 Temp, B4 Temp, B5 Temp, B6 Temp, Voltage
2015/10/22 21:52:59, 1.873, 21.3, 48.6, 22.3, 49.3, 21.7, 21.7, 21.8, 21.9, 21.8, 22.1, 21.6, 21.4, 21.4, 21.3, 21.4, 21.4, 4.06
First a time stamp, then weight (kg) max. 200 kg with 3 decimal places. 2x a temperature and humidity duo (temp can be negative), then 12 temperature values in the format +-xx.x last volts in x.xx format.
- My first thought was to set encryption aside, but you "earn" only some characters, see https://lowpowerlab.com/forum/index.php/topic,1368.0.html
- Second approach was to use not only one package but more. But there is no lib or lib extension for that so I give up because I can not do this by myself.
- Next was to search for an other lib but I do not see any alternative.
Some may ask why not using a struct: The downside of the struct approach is that the gateway has to know what data and type is coming in. So you can not use
- nodes with changing payload without updating the gateway also
- different nodes and the same gateway sketch. The only approach would be to filter by node ID and have different functions for each type of node
What options are left?
sending not one but many packages
=> this is not a good idea without control overhead because a package can be lost and the whole data structure is damaged
sending not only one package but 19 packages, each with time stamp and variable identifier e.g.:
- 2015/10/22 21:52:59, Weight,1.873
- 2015/10/22 21:52:59, Outside Temp, 21.3
- 2015/10/22 21:52:59, Outside Humid, 48.6
=> lot investment of time, bandwidth and power
sending parts and label parts to build it together on the gateway side
2015/10/22 21:52:59, part1, 1.873, 21.3, 48.6, 22.3, 49.3, 21.7
2015/10/22 21:52:59, part2, 21.7, 21.8, 21.9, 21.8, 22.1, 21.6,
2015/10/22 21:52:59, part3, 21.4, 21.4, 21.3, 21.4, 21.4, 4.06
=> don't know how solid this can be done
using struct, with all disadvantages, and try to compress variables additionally, what can I do?
- int 213 instead of char (or float) 21.3 makes 2 bytes vs. 4 bytes
- I save the place for delimiter
- using unix timestamp not ascii date/time
Any other thing to do in optimizing the struct? Or any other idea for big data payloads?
I suggest using the struct, but adopt a standardised layout for the preamble of each packet that includes an identufier for what kind of struct it is.
A "message type" field if you like. Put it in every message in the same place so your decode logic knows how to interpret the rest of the payload. That way you can add new stuff to your network later without changing the existing nodes.
Quote from: Clemens on October 22, 2015, 05:22:49 PM
I have a payload / data sets with about 130 character and I wonder what opportunities I have to transfer the data via RFM96 modules:
This is a demo data set and description:
Date/Time, Weight, Outside Temp, Outside Humid, Inside Temp, Inside Humid, H1 Temp, H2 Temp, H3 Temp, H4 Temp, H5 Temp, H6 Temp, B1 Temp, B2 Temp, B3 Temp, B4 Temp, B5 Temp, B6 Temp, Voltage
2015/10/22 21:52:59, 1.873, 21.3, 48.6, 22.3, 49.3, 21.7, 21.7, 21.8, 21.9, 21.8, 22.1, 21.6, 21.4, 21.4, 21.3, 21.4, 21.4, 4.06
It seems to me you have an inefficient packet structure. You use commas and spaces which waste a lot of bytes and a very long datetime format.
You could settle on a single separator, and a unix timestamp:
1445600261 instead of
2015/10/22 21:52:59 which is almost double in size. If you would use structs I'm sure all this data could fit in 61 bytes.
Structures are the way to go to maximize efficiency. As long as both sides agree on endianess and formats of floats and the structures are packed, one possibility would be as follows:
Time -- 4 byte unix seconds since epoch * 1 = 4 bytes
Weight -- 4 byte float * 1 = 4 bytes
Temperature -- 4 byte float * 8 = 32 bytes
Humidity -- 1 byte unsigned char scaled to 100% = 255 (gives a precision of about .4%, doubt you can measure humidity this precisely) * 8 = 8 bytes
Voltage -- 4 byte float * 1 = 4 bytes
Adds up to 52 Bytes. Add an 1-2 byte identifier per @damonb and you even have room for expansion.
All my temperatures are send as int16_t with a simple trick : all values is multiplied by 10 that allows me to have temperature between -3267,8°C and 3276,7°C (that should be enough for a while). If you really need more precision you can always multiply it by 100. (I never needed that).
So 2 bytes are enough for me.
Most of my moteinos are sending 9 bytes :
* 2 * 2 bytes for temperature (I got 2 DS18B20 each time)
* 1 byte for humidity
* 2 bytes for battery voltage
* 2 bytes for luminosity (calculated with a cheap LDR)