Author Topic: Send WAY Fewer Bytes  (Read 18979 times)

ChemE

  • Sr. Member
  • ****
  • Posts: 419
  • Country: us
Send WAY Fewer Bytes
« on: December 14, 2016, 02:15:33 PM »
I'm getting ready to start on a personal project for the challenge/fun/curiosity/learning that most would consider foolhardy which will involve cutting down Felix's radio code significantly to produce a variant that really only knows how to initialize the radio and send a transmission.  In looking at the standard RFM69 library it seems that even if a TH node tries to only broadcast 2 bytes of data...you actually send 66 bytes of data which is obviously uses way the heck more battery than is needed.  I know it is being done to take advantage of encryption but it still seems exceedingly out of character with the main bent of this community.

Does anyone have some functioning radio code that sends a smaller frame that I can modify from rather than the full blown library?  Preferably one that isn't interrupt driven either since I'm not planning to use interrupts in my project.  I've got to think given some of the lengths others have gone to to increase battery life that sending 12 bytes rather than 70 hasn't been overlooked.

joelucid

  • Hero Member
  • *****
  • Posts: 868
Re: Send WAAAY Fewer Bytes
« Reply #1 on: December 15, 2016, 09:13:05 AM »
Rfm69 uses a variable size packet format and does not always send full size. If you use encryption the payload is sized in 16 bytes chunks though. The minimal overall packet size including overhead is 24 bytes with encryption.

Joe

ChemE

  • Sr. Member
  • ****
  • Posts: 419
  • Country: us
Re: Send WAAAY Fewer Bytes
« Reply #2 on: December 15, 2016, 03:49:31 PM »
My bad, upon closer inspection of the code it does not broadcast as many bytes as I initially thought.  I still plan on messing around with fixed length packets and cutting out some of the other bytes to see what is possible as far as minimal payloads.  It sounds like I'll be the first to do so?

joelucid

  • Hero Member
  • *****
  • Posts: 868
Re: Send WAY Fewer Bytes
« Reply #3 on: December 15, 2016, 06:30:20 PM »
I find that I usually transmit close to full packet size since I like to record a lot of data about my motes. If you want to reduce transmit time going with a higher Bitrate might be more promising. If you have a decent gw with decent antenna you should be able to run at 200 kbit even with rfm69w's throughout the house.

A higher Bitrate also reduces tx times for all the overhead. So you can save much more than by reducing the payload.

Joe

ChemE

  • Sr. Member
  • ****
  • Posts: 419
  • Country: us
Re: Send WAY Fewer Bytes
« Reply #4 on: December 16, 2016, 01:47:16 AM »
I agree that transmitting at the fastest bitrate is clearly step 1 and I'm already transmitting at 300kbs so I am on the air about 1/6th of what the library defaults required.  My data payload is only three bytes though and actually I have worked out a scheme in which I can transmit temperature and humidity in a single byte once the initial temperature and humidity have reached the gateway.  It works like this:

Bit 1: 0 for decrease 1 for increase
Bits 2-4: Encode delta T x10
Bit 5: 0 for decrease 1 for increase
Bits 6-8: Encode delta RH x10

The node keeps the last measured T and RH and if either new result differs from the last transmission by more than 0.1F or 0.1% RH it sends out the differential result according to the scheme above.  I won't be able to get rid of all the preamble bytes but moving from 24 bytes to 1 byte represents a further 96% reduction in time on the air.  Even just sending the two byte temperature measurement and two byte humidity measurement with a few preamble bytes without the two byte CRC, length byte, and address byte would still allow for a 75% reduction in on-air time relative to where I am at currently.  This still feels like low hanging fruit to me I am just surprised that no one else has felt the same.  I'll share my results regardless.
« Last Edit: December 16, 2016, 02:27:52 AM by ChemE »

joelucid

  • Hero Member
  • *****
  • Posts: 868
Re: Send WAY Fewer Bytes
« Reply #5 on: December 16, 2016, 02:45:50 AM »
Quote
My data payload is only three bytes though and actually I have worked out a scheme in which I can transmit temperature and humidity in a single byte once the initial temperature and humidity have reached the gateway.

Oh - I'm sorry, clearly I've misjudged your position on the learning curve based on your packet size confusion  ;). What you're doing does sound promising!

If you switch off AGC and always use max gain you can likely reduce preamble size to two bytes because rx startup time is faster that way. You could use a 1 byte sync word. You likely do need some form of error detection. If you only send one byte a couple of bits might be enough instead of the full CRC16. You probably want to enable nodes other than TH so you'd need some msg identifier in your payload. This requirement might also mean adding a length byte to enable variable length packets.

That takes you to what - 6-7 bytes. Yeah that is much smaller than what I send.

I've done some research and implementation on another approach to reduce TX time for TH motes which seemed really promising: predictive modeling. I thought I had started a whole thread on it here complete with pointers to science papers but it seems all gone. The only thing I found is a summary from an email:

Quote
If this isn’t good enough one could do predictive th coding: Notice how the t curve is linear over shorter periods. That’s to be expected since it’s driven by energy transfer due to temperature differences or heating which transfer a near constant energy amount per time period.
 
I bet you could model the temperature over a period of say an hour as second order polynomial fairly well. Then you send its parameters along with the current temperature on every update. Updates only get send if the model needs to be updated. Otherwise the server can infer the temperature from the last measurement. The model could get better over time – learning how the heating system is set up in a room for example.
 
With that approach I bet you could comfortably hit the 50 year boundary.

It's really weird that this thread is gone. I did some tests and this worked pretty well.

Joe

ChemE

  • Sr. Member
  • ****
  • Posts: 419
  • Country: us
Re: Send WAY Fewer Bytes
« Reply #6 on: December 16, 2016, 03:52:27 AM »
Huh, inferred values based on slope is an interesting notion, that has not occurred to me.  A whole byte for address certainly is overkill in any system I can envision myself deploying.  If it comes to it, I might set aside a few bits in the payload to act as an address.  Thanks for the tip on the AGC.  I've asked Santa for a one channel oscilloscope for Christmas so I can hopefully measure just how long my radio is actually broadcasting rather than using calculations in Excel based on bitrate and packet size.  Speaking of which, my current TH node power budget looks like this:



Those calculated battery times are in years and neglect self discharge.  From this I clearly would benefit from ditching the WDT on the 328p and switching to my TLP5110 to wake the uC but one thing at a time.  I want to squeeze the radio a little more before I move on. :)

joelucid

  • Hero Member
  • *****
  • Posts: 868
Re: Send WAY Fewer Bytes
« Reply #7 on: December 16, 2016, 04:22:22 AM »
Quote
Those calculated battery times are in years and neglect self discharge.  From this I clearly would benefit from ditching the WDT on the 328p and switching to my TLP5110 to wake the uC but one thing at a time.  I want to squeeze the radio a little more before I move on. :)

Clearly the sleep current dominates this picture. You can use the the radio as wakeup device and get it down to ~1.3uA - biggest bang for the buck. Just use listen mode, map PLLLocked to DIO0, sleep without WDT and let PLLLocked wake you up. Then quickly switch the radio off so you don't waste energy in RX.

ChemE

  • Sr. Member
  • ****
  • Posts: 419
  • Country: us
Re: Send WAY Fewer Bytes
« Reply #8 on: December 16, 2016, 04:33:49 AM »
True given a Moteino.  This foolhardy project of mine will eventually be an ATTiny85, an AB1815, a Si7021, and a RFM69W all on a nice little PCB from OSH Park.  The hex file size and pin count do not appear to be insurmountable at all.  Once that works then I will have a nice little platform upon which to build other cool things.
« Last Edit: December 16, 2016, 04:44:00 AM by ChemE »

joelucid

  • Hero Member
  • *****
  • Posts: 868
Re: Send WAY Fewer Bytes
« Reply #9 on: December 16, 2016, 07:14:42 AM »
Quote
This foolhardy project of mine will eventually be an ATTiny85, an AB1815, a Si7021, and a RFM69W all on a nice little PCB from OSH Park.  The hex file size and pin count do not appear to be insurmountable at all.

Ah ok, interesting. BTW the one thing that's discouraged me a bit from the "iifetime mote" concept (essential a mote that would survive you) is my experience with humidity sensors. Somehow today I can't imagine a si7021 to still reliably measure humidity after 10 years in use.

If one accepts 10 years of reasonable life then the avr + rfm69w + 0,1uF cap + cr2032 combo provides a super simple combination to get there.

Joe

ChemE

  • Sr. Member
  • ****
  • Posts: 419
  • Country: us
Re: Send WAY Fewer Bytes
« Reply #10 on: December 16, 2016, 07:49:51 AM »
Part of my motivation here besides the programmatic challenge of squeezing it all onto 6 pins and 8kb is I have 0 skills when it comes to SMD solder reflow and PCB design so it is also I good learning exercise for me.

WhiteHare

  • Hero Member
  • *****
  • Posts: 1300
  • Country: us
Re: Send WAY Fewer Bytes
« Reply #11 on: December 16, 2016, 11:44:14 AM »
Part of my motivation here besides the programmatic challenge of squeezing it all onto 6 pins and 8kb is I have 0 skills when it comes to SMD solder reflow and PCB design so it is also I good learning exercise for me.

If that's your fear, then atmega328p comes in a dip format: atmega328p-pu.  There's really no longer a meaningful cost savings to justify going to an attiny85, if that's the other factor driving your decision.

However, you'll probably find that hand soldering the SMD atmega328p's isn't as daunting as it looks if you use Dave Jones's "Tack and reflow" method of hand soldering.  Buy some $1 atmega328p's from China and give it a try.  Even if you completely FUBAR it, you can always just re-do it.
« Last Edit: December 16, 2016, 11:49:57 AM by WhiteHare »

ChemE

  • Sr. Member
  • ****
  • Posts: 419
  • Country: us
Re: Send WAY Fewer Bytes
« Reply #12 on: December 16, 2016, 12:48:59 PM »
If that's your fear, then atmega328p comes in a dip format: atmega328p-pu.  There's really no longer a meaningful cost savings to justify going to an attiny85, if that's the other factor driving your decision.

Thanks, I actually bought 10 328p's in dip format from Mouser a long time ago and I use them for projects that I gift to people or deploy into the wild (my house/yard/garage).  No, the $1 saved in moving from a 328p to a Tiny85 isn't my motivation at all since the time I'll spend on this project is worth vastly more than that if this were a commercial endeavor.  It is my love of doing more with less that is the motivation here.  Plus, small lean code executes faster and therefore saves battery. 

Regarding an update to the thread itself, I'm getting good results with 8-byte packets which have 4-byte payloads (this done to shift float point math off the node and therefore shave a TON of hex file size).  0 preamble bytes, the normal 2 sync bytes, 1 length byte, 1 address byte, and then my 4-byte payload is working fine with no CRC.  I still need to do more experimentation but a 66% reduction in a morning isn't bad.

joelucid

  • Hero Member
  • *****
  • Posts: 868
Re: Send WAY Fewer Bytes
« Reply #13 on: December 16, 2016, 01:13:36 PM »
Quote
0 preamble bytes

Interesting that that works. Rssi detection happens during preamble reception before packet reception starts. So my guess is that your RSSI threshold is so low that the radio starts reception way before the actual packet is sent. Then the radio is ready in time to recognize the sync word.

ChemE

  • Sr. Member
  • ****
  • Posts: 419
  • Country: us
Re: Send WAY Fewer Bytes
« Reply #14 on: December 16, 2016, 01:56:49 PM »
The two radios antennas are currently one whole inch away from each other to make it easy to upload node and gateway code for each trial; that probably has something to do with it as well!  I would expect having to add back a preamble byte when I move my node further away and doing so has almost zero cost relative to my power budget.  Still, why send 9 bytes when 8 will do?

« Last Edit: December 16, 2016, 02:00:45 PM by ChemE »