Author Topic: How to speed up RFM69 data throughput on Moteino M0? [solved]  (Read 42006 times)

ChemE

  • Sr. Member
  • ****
  • Posts: 419
  • Country: us
Re: How to speed up RFM69 data throughput on Moteino M0?
« Reply #15 on: March 27, 2020, 06:37:32 AM »
I'm going back through the code and reminding myself where you'll need to rewrite to avoid the SPI session thrashing.  The changes you need to make are in RFM69.cpp around line 437 in RFM69::select and RFM69::unselect

Here is Felix's original code:
Code: [Select]
// select the RFM69 transceiver (save SPI settings, set CS low)
void RFM69::select() {
  noInterrupts();
#if defined (SPCR) && defined (SPSR)
  // save current SPI settings
  _SPCR = SPCR;
  _SPSR = SPSR;
#endif
  // set RFM69 SPI settings
  SPI.setDataMode(SPI_MODE0);
  SPI.setBitOrder(MSBFIRST);
  SPI.setClockDivider(SPI_CLOCK_DIV4); // decided to slow down from DIV2 after SPI stalling in some instances, especially visible on mega1284p when RFM69 and FLASH chip both present
  digitalWrite(_slaveSelectPin, LOW);
}

// unselect the RFM69 transceiver (set CS high, restore SPI settings)
void RFM69::unselect() {
  digitalWrite(_slaveSelectPin, HIGH);
  // restore SPI settings to what they were before talking to RFM69
#if defined (SPCR) && defined (SPSR)
  SPCR = _SPCR;
  SPSR = _SPSR;
#endif
  maybeInterrupts();
}

It is easy enough to just comment out the code that caches and restores the SPI bus settings like this:
Code: [Select]
void RFM69::select() {
  noInterrupts();
//#if defined (SPCR) && defined (SPSR)
  // save current SPI settings
  //_SPCR = SPCR;
  //_SPSR = SPSR;
//#endif
  // set RFM69 SPI settings
  //SPI.setDataMode(SPI_MODE0);
  //SPI.setBitOrder(MSBFIRST);
  //SPI.setClockDivider(SPI_CLOCK_DIV4); // decided to slow down from DIV2 after SPI stalling in some instances, especially visible on mega1284p when RFM69 and FLASH chip both present
  digitalWrite(_slaveSelectPin, LOW);
}

// unselect the RFM69 transceiver (set CS high, restore SPI settings)
void RFM69::unselect() {
  digitalWrite(_slaveSelectPin, HIGH);
  // restore SPI settings to what they were before talking to RFM69
//#if defined (SPCR) && defined (SPSR)
  //SPCR = _SPCR;
  //SPSR = _SPSR;
#endif
  maybeInterrupts();
}

Also it is worth noting that Arduino's digitalWrite() function is an absolute dog and manages to waste like 113 clock cycles doing what can be done in 1 clock cycle.  If you want to save even more time, avoid digitalWrite everwhere in your code and just bang bits instead.
Code: [Select]
#define 	SS_PIN			PB2
#define    SS_WRITE_LOW      PORTB &= ~(1<<SS_PIN)
#define    SS_WRITE_HIGH      PORTB |= 1<<SS_PIN
Much more on bit banging here: https://lowpowerlab.com/forum/general-topics/super-sweet-bit-banging-macros/
I'll be curious to see how this changes your refresh rate!
« Last Edit: March 27, 2020, 06:44:12 AM by ChemE »

jdheinzmann

  • NewMember
  • *
  • Posts: 40
  • Country: us
Re: How to speed up RFM69 data throughput on Moteino M0?
« Reply #16 on: March 27, 2020, 06:04:39 PM »
Wow, thanks, ChemE.  I will try that.  And I will keep the digitalWrites in mind.  Presently, I am only using them for studying timing with the scope.

Meanwhile, I went hunting through my transmit side code for opportunities.  First, I found that when I commented out some string manipulations for Serial Monitor debug printing, my throughput got worse!!!  A little scoping of both sides at once revealed that I was attempting two sends for each receive cycle.  I found that the next send needed to start at least 550 us after the present receiveDone() became true.  I don't exactly know why this is but I can live with that constraint. 

So I played around with some delay on the send side and found that I could get the throughput up from 318 to about 420 Hz which is another good step in the right direction for me.

Thanks, again, for your help.  -JD

jdheinzmann

  • NewMember
  • *
  • Posts: 40
  • Country: us
Re: How to speed up RFM69 data throughput on Moteino M0?
« Reply #17 on: March 27, 2020, 07:20:55 PM »
Okay.  So I tried ChemE's suggestions for modifying the SPI select code.  Without trying to change the digitalWrites, I saw a reduction in send time from about 1.5 ms, to 1.4 ms.  This is good!

I further tried to replace the digitalWrites.  Since PB2 is not defined I tried using _slaveSelectPin instead.  Then the compiler complained with:

Code: [Select]
error: no match for 'operator&=' (operand types are '_EPortType' and 'int')
Also, I don't see PORTB defined anywhere.  Perhaps that is the problem?

ChemE

  • Sr. Member
  • ****
  • Posts: 419
  • Country: us
Re: How to speed up RFM69 data throughput on Moteino M0?
« Reply #18 on: March 28, 2020, 10:49:55 AM »
Looks like the slave select pin on the MoteinoM0 is PB09.  Part of the problem is my code is written for and specific to an Atmel 328p because that is all I use and I don't care about portability from one uC to another.

Felix

  • Administrator
  • Hero Member
  • *****
  • Posts: 6866
  • Country: us
    • LowPowerLab
Re: How to speed up RFM69 data throughput on Moteino M0?
« Reply #19 on: March 28, 2020, 09:09:38 PM »
ChemE's defines are very cool but are for AVR, will not work on SAMD  :'(

For SAMD21 and SAMD51 you can use direct register manipulation to flip bits with the defines shown below.
Notice how you can just toggle a bit without remembering its state like it's typically done on AVRs ;-)

For instance this sample sketch below toggles the MoteinoM0 LED_BUILTIN every second (arduino pin 13, aka pin PA17).
Let us know how fast this makes your code now!  :o

Code: [Select]
#define LED_PIN PIN_PA17

#define PINOP(pin, OP) (PORT->Group[(pin) / 32].OP.reg = (1 << ((pin) % 32)))
#define PIN_OFF() PINOP(LED_PIN, OUTCLR)
#define PIN_ON() PINOP(LED_PIN, OUTSET)
#define PIN_TGL() PINOP(LED_PIN, OUTTGL)

void setup() {
  PINOP(LED_PIN, DIRSET);
}

void loop() {
  PIN_TGL();
  delay(1000);
}

I scoped out the code below and I get these results:
- SAMD21 - ~100ns pulse with the toggles and a  ~1530ns pulse with digitalWrite
- SAMD51 - ~20ns pulse with the toggles and a  ~400ns pulse with digitalWrite

Code: [Select]
void loop() {
  PIN_TGL();
  PIN_TGL();
  delayMicroseconds(1);
  digitalWrite(13,HIGH);
  digitalWrite(13,LOW);
  delay(2);


« Last Edit: March 28, 2020, 09:47:24 PM by Felix »

ChemE

  • Sr. Member
  • ****
  • Posts: 419
  • Country: us
Re: How to speed up RFM69 data throughput on Moteino M0?
« Reply #20 on: March 28, 2020, 09:42:38 PM »
Awesome, glad you had the solution for the SAM21D Felix!  I'm also curious to see how fast this gets the refresh rate.  Are you at all close to your 1000Hz target yet?

jdheinzmann

  • NewMember
  • *
  • Posts: 40
  • Country: us
Re: How to speed up RFM69 data throughput on Moteino M0?
« Reply #21 on: March 29, 2020, 06:08:29 PM »
Hi, Felix.  After some fussing I figured out what I needed to do to incorporate your suggested macros into the RFM69 library.  And it works! 

The radio send time is now further reduced from 1.40 ms to 1.34 ms.  With that came an increase in throughput from about 420 to to 434 Hz.  Even better!  However, I noticed that after about 11 seconds of running, throughput started faltering with dropped receives every now and then.  I had to increase the delay in my sketch from 250 to 270 µs to stop this so my throughput is now 427 Hz.  Still, this is better, a little bit anyway. 

Here is the code with the SPI save and restore steps commented out and the digital writes replaced with the PIN_OFF and PIN_ON macros.

Code: [Select]
// Modification in Felix's code to remove SPI "thrashing" and speed up digitalWrites - JD
#define SS_PIN            PIN_PB09
#define SS_PINOP(pin, OP) (PORT->Group[(pin) / 32].OP.reg = (1 << ((pin) % 32)))
#define SS_PIN_OFF()      SS_PINOP(SS_PIN, OUTCLR)
#define SS_PIN_ON()       SS_PINOP(SS_PIN, OUTSET)
//#define SS_PIN_TGL() SS_PINOP(SS_PIN, OUTTGL)

// select the RFM69 transceiver (save SPI settings, set CS low)
void RFM69::select() {
//#if defined (SPCR) && defined (SPSR)
  // save current SPI settings
  //_SPCR = SPCR;
  //_SPSR = SPSR;
//#endif

#ifdef SPI_HAS_TRANSACTION
  SPI.beginTransaction(_settings);
#endif

  // set RFM69 SPI settings
  //SPI.setDataMode(SPI_MODE0);
  //SPI.setBitOrder(MSBFIRST);
#ifdef __arm__
//SPI.setClockDivider(SPI_CLOCK_DIV16);
SPI.setClockDivider(SPI_CLOCK_DIV2); // JD - trying to speed up SPI clock
#else
  SPI.setClockDivider(SPI_CLOCK_DIV4); // decided to slow down from DIV2 after SPI stalling in some instances, especially visible on mega1284p when RFM69 and FLASH chip both present
#endif
  //digitalWrite(_slaveSelectPin, LOW);
  SS_PIN_OFF();
}

// unselect the RFM69 transceiver (set CS high, restore SPI settings)
void RFM69::unselect() {
  //digitalWrite(_slaveSelectPin, HIGH);
  SS_PIN_ON();
#ifdef SPI_HAS_TRANSACTION
  SPI.endTransaction();
#endif 
  // restore SPI settings to what they were before talking to RFM69
//#if defined (SPCR) && defined (SPSR)
  //SPCR = _SPCR;
  //SPSR = _SPSR;
//#endif
}
// End of code modified to avoid SPI "thrashing" - JD
Have I done this correctly?  Anything else I should be doing?

ChemE, I originally said I wanted to increase throughput to 1 kHz.  I was dreaming.  I was forgetting that in reality, 400 Hz is good enough for now.  It allows me to still send 100 scans per revolution of my rotating system at 240 RPM.  At speeds below 120 rpm, I can send 200 scans per revolution which is my preferred rate at lower speeds for best resolution of the plots and animations I do with this data.

Unless you all have any further suggestions, I will mark this solved.
« Last Edit: March 29, 2020, 06:10:17 PM by jdheinzmann »

Felix

  • Administrator
  • Hero Member
  • *****
  • Posts: 6866
  • Country: us
    • LowPowerLab
Re: How to speed up RFM69 data throughput on Moteino M0?
« Reply #22 on: March 29, 2020, 07:03:15 PM »
Unless you all have any further suggestions,
Oh!
Tell us what you're doing transmitting at 400hz with a MoteinoM0!  ;D

And how large in bytes are your packets on average?

ChemE

  • Sr. Member
  • ****
  • Posts: 419
  • Country: us
Re: How to speed up RFM69 data throughput on Moteino M0?
« Reply #23 on: March 29, 2020, 08:59:33 PM »
Unless you all have any further suggestions, I will mark this solved.

I still see some session stuff hanging out but you've probably gotten most of the fat trimmed away at this point.  If your life depended on it, you could probably still up the rate 50% by cutting out all those function calls and returns, ditching the last of the SPI session stuff, etc.  But, if it is already fast enough for your use case, then I'd say call it solved.  I'm unusual in that part of my joy of coding is finding that last 1% which most everyone else doesn't care about.

jdheinzmann

  • NewMember
  • *
  • Posts: 40
  • Country: us
Re: How to speed up RFM69 data throughput on Moteino M0?
« Reply #24 on: March 30, 2020, 05:35:36 PM »
Felix, what I am doing is making an inertial based wireless shaft angle encoder for use on large stationary steam engines in an antique machinery museum.  The LDIS3DH accelerometer gives me angle from the X and Y directions without needing a mechanical connection to the stationary frame.  And the radio gives me a data connection to the stationary frame without twisting up any wires.  This sensor can be mounted to the end of the shaft of any one of these large machines using magnetic feet with no mod to the shaft.  That signal along with steam pressure signals from sensors on either end of the cylinder yield real-time PV "indicator" diagrams which is the modern way of measuring indicated horsepower (vs. brake horsepower with a dynamometer).  The area enclosed by the PV cycle is the work done on the piston during that cycle.  The shaft angle (along with known geometry of the engine) is also used in displaying a real time animation of the piston and valve gear.  Steam pressure is used to change the color of the steam inside the engine (lighter for positive pressure, darker for negative).  It works quite nicely already with a wired analog angle sensor on a particular engine and an Arduino with Serial connection to Processing on a PC.  The wireless sensor will make the hook-up so much easier for other engines.  Visitors to the museum really like being able to get a better picture of what is going on inside these engines.



Oh, and my packets are 20 bytes long.  I will be shortening them as there is presently debug info in there that will not be needed much longer. 

ChemE, as far as eliminating SPI thrashing goes, though the throughput was higher, the data integrity suffered.  The first element of the struct passed over gets corrupted (in a rather repeatable way) every now and then.  So I removed that but kept Felix's PIN ON/OFF macros.  Perhaps I hadn't implemented the changes properly.  The code in the present library doesn't exactly match the sample you provided.  Should I have commented these out too?
Code: [Select]
#ifdef SPI_HAS_TRANSACTION
  SPI.beginTransaction(_settings);
#endif
My throughput is still a little above 400 Hz so I remain happy.

(How do I get the attached image to display inline?  I have
Code: [Select]
[img]https://IMG_20190927_154904.jpg[/img]
between the above two paragraphs.)
« Last Edit: March 30, 2020, 05:50:09 PM by jdheinzmann »

Felix

  • Administrator
  • Hero Member
  • *****
  • Posts: 6866
  • Country: us
    • LowPowerLab
Re: How to speed up RFM69 data throughput on Moteino M0?
« Reply #25 on: March 30, 2020, 08:57:10 PM »
JD,
Thanks for sharing the background,
Very fascinating, so great that the Moteino M0 can do the job.

When you add an attachment, unfortunately it will go at the bottom. If you want to insert an image in the middle it will have to be from an online location.
OR if you first post the post+attachment then copy the link of the posted image and edit your post with an additional IMG tag with the link in it:

https://lowpowerlab.com/forum/moteino-m0/best-way-to-speed-up-rfm69hcw-data-throughput-on-moteino-m0/?action=dlattach;attach=3276

jdheinzmann

  • NewMember
  • *
  • Posts: 40
  • Country: us
Re: How to speed up RFM69 data throughput on Moteino M0?
« Reply #26 on: April 03, 2020, 05:16:43 PM »
As I try to streamline radio communications, I've added conditional compiles to leave out code needed or not needed for development vs. fast running.

I tried removing the nodeId element from the Payload struct and got rather strange behavior.

Code: [Select]
// constants for specifying mode in which program is to run
#define CAL            (0) // run sketch in calibration mode
#define DEV            (1) // debug data included in packets over radio
#define RUN            (2) // only essential data in packets over radio

#define RUN_MODE       (RUN)  // CAL, DEV or RUN

// Set up structure for radio packet payload. 
// Both sides must have exact same structure to send from and receive into.
#if (RUN_MODE == RUN)
  typedef struct
  {
    float              theta; // [rad], shaft angle
    float               batV; // [V], wireless sensor battery voltage
    boolean              TDC; // true when TDC has been reached, otherwise false
  } Payload;
#else // must be CAL or DEV
  typedef struct
  {
    int               nodeId; // store this nodeId
    unsigned long remTime_ms; // [ms], elapsed time
    float              theta; // [rad], shaft angle
    float                  X; // [g], acceleration in X direction
    float                  Y; // [g], acceleration in Y direction
    float               batV; // [V], wireless sensor battery voltage
    boolean              TDC; // true when TDC has been reached, otherwise false
  } Payload;

Payload RFDataPktToSend;
#endif
When I try to run this, the code hangs up while reading the accelerometer (over SPI) with no error message.  In loop(), it just never makes it past the first read of the accelerometer (as evidenced by a Serial.print on either side of the read.  Only the first one prints then nothing.)
Code: [Select]
  Serial.println("About to read IMU..."); // debug
  X_Raw = myIMU.readFloatAccelX();
  Serial.println("Read X from IMU..."); // debug
If I change to RUN_MODE (DEV), it runs fine.

FYI, this is the send (though it never gets there):
Code: [Select]
      radio.send(GATEWAYID, (const void*)(&RFDataPktToSend), sizeof(RFDataPktToSend));    

1)  I just want to confirm that NodeId does not need to be in the struct for the radio to function, right?

2)  From experimentation, it seems that the Payload struct must start with an int or a long.  Otherwise the sketch will not run.  Any reason that you know of for this?  I mean, an int or a float are both just 4 bytes of binary data.  What does it care?  Hmm...

Any ideas?

jdheinzmann

  • NewMember
  • *
  • Posts: 40
  • Country: us
Re: How to speed up RFM69 data throughput on Moteino M0?
« Reply #27 on: April 03, 2020, 09:37:15 PM »
On the brighter side, with my trimming down and streamlining I am now seeing 500 Hz throughput.  Woo hoo!

I got here by removing all Serial.print()s on the transmit side.  Much of the time there is spent in assembling the strings to be printed.  Using sprintf to assemble the strings added 1110 µs while stringing out a bunch of Serial.print()s (11 of them) adds 910 µs instead.  On the receive side, I can't completely get rid of the Serial.print()s because I need to send the data gathered from both the remote and local sensors up to the PC.  But I did switch to binary instead of CSV data.  When I activate serial monitor though, the throughput rate slows down quite a bit and the updates jitter a lot.  Hopefully I will be able to do better when the data is going to Processing instead.

One problem I still see though is that the radio.send()s, which typically take 1.33 ms, occasionally take upwards of 9 ms.  By occasionally, I mean once about every 2.5 s, and there is usually a burst of them typically lasting 38 to 72 ms.  What could be causing this?  I am measuring this timing on a scope using digitalWrites immediately before and after each radio.send().    Remember, these sends are without ACKs, and without encryption.  The receive side time spent servicing each received data packet is taking a stable 520 µs, so it does not appear to be anything on the receive side that is causing this.  Could this be radio interference?  Would the send side feel this?  Without ACKs/retries?  What could be stretching out these sends?

Edit:  I forgot to mention that the above change to the Payload struct reduced the size from 28 to 16 bytes.  (I should also note that in reply #24, I had mistakenly reported the Payload as being 20 bytes.  It was 28.)
« Last Edit: April 03, 2020, 10:40:29 PM by jdheinzmann »

ChemE

  • Sr. Member
  • ****
  • Posts: 419
  • Country: us
Re: How to speed up RFM69 data throughput on Moteino M0?
« Reply #28 on: April 04, 2020, 08:06:05 AM »
I would hope the only things your transmit Mote is doing are:
1) getting numeric data from your accelerometer
2) assembling the numeric data into a radio packet
3) broadcasting the packet

If it is doing any string manipulation at all, you are slowing it down.

jdheinzmann

  • NewMember
  • *
  • Posts: 40
  • Country: us
Re: How to speed up RFM69 data throughput on Moteino M0?
« Reply #29 on: April 04, 2020, 10:05:26 AM »
Hi, ChemE.  Yes, that is all it is doing when in RUN mode.  When in DEV mode it assembles debug strings and writes them to USB, and yes, it runs slower then.  -JD