most tightly packed stream of packets?

[WhiteHare]

I want to send a tightly packed stream of packets for the purpose of hitting a small Rx window in Listen Mode.  This is the best I've come up with so far (the SPI code is slightly altered compared to the RFM69 library code):

Code Select Expand

void sendQuickPacketStream() {
  while (digitalRead(4)) {//while button is pushed
    radio.writeReg(REG_OPMODE,B00001000); //activate RFM69 FS mode. 

    //Load FIFO
    SPI.setDataMode(SPI_MODE0);
    SPI.setBitOrder(MSBFIRST);
    SPI.setClockDivider(SPI_CLOCK_DIV4); 
    digitalWrite(10, LOW);  //select();

    SPI.transfer(REG_FIFO | 0x80);
    
    SPI.transfer(4);  ////SPI.transfer(sendSize + 3);
    SPI.transfer(GATEWAYID);   //SPI.transfer(toAddress);  
    SPI.transfer(NODEID);  // SPI.transfer(fromAddress);
    SPI.transfer(0x00);  //CTLbyte
    SPI.transfer((payload[0]));  //just a one byte payload
  
    digitalWrite(10, HIGH);      //unselect();

    while (digitalRead(2)) {}  //guarantee any prior Tx mode exited after PacketSent

    radio.writeReg(REG_OPMODE,B00001100); //activate RFM69 Tx mode. 
    while (!(digitalRead(2))) {}  //busy-wait until PacketSent

  }

  //Assertion: button is released
  blockingActivateStandbyMode();
}

DIO0 is mapped to D2 using DIOx setting 00 (see table 22 of DS).

Is this as tightly packed as the packet stream can get?  I had also tried just leaving the RFM69 in Tx mode after PacketSent and then loading the next packet into the FIFO, but doing that doesn't transmit the next packet.  It seems that one must exit and then re-enter Tx mode after PacketSent in order to transmit the next packet.  So, I'm exiting to FS mode so that the FS stays revved up.

[perky]

Dunno, I wonder whether you can use auto mode:

Enter condition set to FIFO not empty
Intermediate condition set to TX
Exit condition set to packet sent.

Now what happens if you start in TX mode so that the state it will exit from intermediate state to is also TX? Does this mean you can stream data packets constantly into the FIFO using almost full to throttle without the FIFO clearing as it would never actually leave TX mode (which would of course clear the FIFO)?

Mark.

[WhiteHare]

That's a clever idea!  If it worked, it would save all that time lost from continually reloading the FIFO with the same packet.  Definitely worth a try.

If it doesn't work, then the next step I was contemplating was to stick the first few bytes in the FIFO, launch Tx mode, and while that gets going I finish loading up the remaining bytes (as suggested by the DS for how to handle extra long packets).  That would give at least some pipelining.

[perky]

If that actually worked it would mean you could start loading in the next packet before the previous one has been sent. The problem with exiting TX at any point is the FIFO gets cleared. You will need to reload in the next packet though, the FIFO gets emptied as it transmits, but you would be constantly pipelining so won't waste any time by doing that.

Mark.

[ChemE]

For certain you can enter TX on FIFO not empty and load a short packet while the radio is spinning up.  At first I played safe and set it to kick on after a few bytes but quickly tested 0 bytes with no problems.

[joelucid]

WH, we've covered this stuff quite a bit. For listen mode you need a continuous signal. So TX needs to be on between packets. That works as long as you don't use encryption and don't use packetsent as indicator (which will get set after the first packet is out, use fifo size instead). There was some success with encryption if the fifo is kept full enough at all times (https://lowpowerlab.com/forum/rf-range-antennas-rfm69-library/burst-message-weirdness-with-listen-mode-(solved-bad-radio)/) although I haven't tried to replicate that.

[WhiteHare]

I haven't yet tried Perky's idea (which I definitely want to do), but based on the helpful comments from ChemE (Tx can be started with an empty FIFO and the payload added to the FIFO afterward) and Joe (staying in TX mode and using FIFO size rather than PacketSent to judge when to start sending the next packet), I did a quick change to my prior code, and it now does function with TX running continuously and not exiting TX mode between packets:

Code Select Expand

void sendQuickPacketStream() {
  radio.writeReg(REG_OPMODE,B00001100); //activate RFM69 Tx mode. 

  while (digitalRead(4)) {//while button is pushed
    SPI.setDataMode(SPI_MODE0);
    SPI.setBitOrder(MSBFIRST);
    SPI.setClockDivider(SPI_CLOCK_DIV4); 
    digitalWrite(10, LOW);  //digitalWrite(_slaveSelectPin, LOW);

    SPI.transfer(REG_FIFO | 0x80);
    
    SPI.transfer(4);  ////SPI.transfer(sendSize + 3);
    SPI.transfer(GATEWAYID);   //SPI.transfer(toAddress);  
    SPI.transfer(NODEID);  // SPI.transfer(fromAddress);
    SPI.transfer(0x00);  //CTLbyte
    SPI.transfer((payload[0]));
    digitalWrite(10, HIGH);    //unselect();

    while ((radio.readReg(REG_IRQFLAGS2)) & B01000000) {} //busy-wait until FIFO is empty
  }
  //Assertion: button is released
  blockingActivateStandbyMode();
}

I suspect this part:

Code Select Expand

    while ((radio.readReg(REG_IRQFLAGS2)) & B01000000)

is slow, as compared to checking a FifoNotEmpty pin that's mapped to from DIO1 or DIO2.  The latter would require a bodge wire, but I've done bodge wires on the RFM69 before, and it's easy to do.

The above code does assume that the FIFO needs to empty before loading the next packet.  It would obviously be preferable if the next payload could be loaded into the FIFO before the packet currently being sent is finished transmitting.  IF that worked (?), then I would only need to avoid overflowing the FIFO.

[Edit1: I just now did a preliminary test, and it looks as though I probably can load the FIFO with the next n payloads prior to the current packet finishing its transmission.  So, I'll now try doing a full coding of that which will keep the FIFO full without overflowing it.  That should, in theory, give the tightest spacing between transmitted packets.]

[Edit2: And if *that* works, then the only thing left to look at (that I can think of) would be the ramp down time after a packet transmits.  Perhaps that can be cut to either zero or near zero.  Doing that should tighten the spacing between packets even further.]

[perky]

Why have you got SPI_CLOCK_DIV4 for your clock divider? You need this clock to be as fast as possible, how about SPI_CLOCK_DIV2?
Mark.

[WhiteHare]

You're right.  Good catch.  I did it that way only because the RFM69 library did it that way.  I think it's to account for somewhat gimpy add-on flash memory (?), which isn't currently present on my remote prototype.

I'll change it.   

[Edit: scratch that.  It doesn't seem to work using DIV2.  I have no theories, so I'm simply reverting back to DIV4]

[WhiteHare]

In theory, by staying in Tx mode and not allowing the FIFO to ever empty, the following code should give rise to a tight packing of transmitted packets:

Code Select Expand

void addPayloadToFifo () {
    SPI.setDataMode(SPI_MODE0);
    SPI.setBitOrder(MSBFIRST);
    SPI.setClockDivider(SPI_CLOCK_DIV4); 
    //SPI.setClockDivider(SPI_CLOCK_DIV2); 
    digitalWrite(10, LOW);  //digitalWrite(_slaveSelectPin, LOW);

    SPI.transfer(REG_FIFO | 0x80);
    
    SPI.transfer(4);  ////SPI.transfer(sendSize + 3);
    SPI.transfer(GATEWAYID);   //SPI.transfer(toAddress);  
    SPI.transfer(NODEID);  // SPI.transfer(fromAddress);
    SPI.transfer(0x00);  //CTLbyte
    SPI.transfer((payload[0]));

    digitalWrite(10, HIGH);  //unselect();
}

void sendQuickPacketStream() {
  radio.writeReg(REG_FIFOTHRESH,B10101000);  //set FIFO threshold to an arbitrary 40 (???)
  radio.writeReg(REG_OPMODE,B00001100); //activate RFM69 Tx mode. 

  while (digitalRead(4)) {//while button is pushed
    while ((radio.readReg(REG_IRQFLAGS2))& B00100000) {} //busy-wait while FifoLevel flagged
    delayMicroseconds(80);  //??? not sure what's the best number to use
    addPayloadToFifo ();
  }
  //Assertion: button is released
  blockingActivateStandbyMode();
}

It contains a couple magic numbers though.  It uses 80 microseconds delay between SPI burst transfers to the FIFO.  I arrived at that through trial and error, and there's probably a better number to use.  For sure, if there's no  delay, it doesn't work.  That at least makes sense.  Strangely, though, if there's too much delay, it doesn't work either!  At least so far, that part doesn't make sense to me.

The other magic number is 40, which I picked arbitrarily for the FifoThreshold used to trigger the FifoLevel flag.  I started with 50, but that led to some packet corruption.  I haven't put any thought yet into picking a principled number. 

This first pass on the code was just to see whether I could get the concept to work at all.  However, as I'm not sure that further fine tuning will actually make any a difference, it may be the final code as well.

[WhiteHare]

After testing it out, though, the prior coding which waits until the FIFO empties before loading the next payload seems to work much better than the latter coding which never lets the FIFO completely empty.     Go figure.

So, for now, I think I'll adopt the previous coding and simply speed it up a bit with a bodge wire.

Also, since the sx1231h allegedly transmits preamble when the FIFO is empty (at least, I seem to recall Joe reporting that), maybe I can transmit a shorter preamble on each packet (that is, if (?) the preamble while waiting is seamless with the explicit packet preamble) and thereby pack the packets a little tighter that way also.  In that case, the waiting time wouldn't be wasted.

[WhiteHare]

OK, I bodge wired DIO2 to pin D7 on the atmega328p, and I added a 10K pulldown resistor to D7 just to be  sure it doesn't float.

Having done that, the following code gives me the tightest packet stream yet:

Code Select Expand

void sendQuickPacketStream() {
  radio.writeReg(REG_OPMODE,B00001100); //activate RFM69 Tx mode. 
  while ((radio.readReg(REG_DIOMAPPING1) != B00001100)) {
    radio.writeReg(REG_DIOMAPPING1, B00001100);
  }
    
  radio.writeReg(REG_DIOMAPPING1, B00000000);  //map DIO2 to FifoNotEmptyFlag
  while ((radio.readReg(REG_DIOMAPPING1) != B00000000)) {
    radio.writeReg(REG_DIOMAPPING1, B00000000); 
  }
  //Note: DIO2 is bodge wired to pin D7 on atmega328p.

  while (digitalRead(4)) { //while button is pushed
    if (!(digitalRead(7))) { //if FIFO is empty
      SPI.setDataMode(SPI_MODE0);
      SPI.setBitOrder(MSBFIRST);
      SPI.setClockDivider(SPI_CLOCK_DIV4); 
      digitalWrite(10, LOW);  //digitalWrite(_slaveSelectPin, LOW);

      SPI.transfer(REG_FIFO | 0x80);
    
      SPI.transfer(4);  ////SPI.transfer(sendSize + 3);
      SPI.transfer(GATEWAYID);   //SPI.transfer(toAddress);  
      SPI.transfer(NODEID);  // SPI.transfer(fromAddress);
      SPI.transfer(0x00);  //CTLbyte
      SPI.transfer((payload[0]));
      digitalWrite(10, HIGH);    //unselect();
    }
  }
  //Assertion: button is released
  blockingActivateStandbyMode();
}

It appears to perform much better than the earlier non-bodged solution.   

[WhiteHare]

With 4 preamble bytes, 3 sync bytes, 2 crc bytes, one length byte, 1 "from address" byte, 1 payload byte, and no encryption, I'm getting acceptable hit rates with a listen window of 1024usec.  That's higher than I was hoping for, and also longer than it seems like it should have to be.  Shorter listen windows "work," but the hit rate noticeably degrades.

The above 11 bytes at 300kbps should take only about 293usec of airtime to transmit.  Therefore, a good listen window of closer to 600usec should be possible, at least theoretically.

That's why I've been pushing for a tighter packet stream.  I'm guessing the gaps between packets must still be large. 

I suppose I may need to do some measurements to figure out what's really going on.

Meanwhile, I'll see if I can get Perky's idea to work.

Alternately, maybe I could trigger the refill of the FIFO to start sooner.  Keeping the FiFo relatively full didn't seem to work very well, but by setting a low enough FifoThreshold, maybe I could pipeline the refill so that it commences within moments of the FIFO becoming empty.  Not sure, but as a last ditch effort, maybe it would be worth a shot.

[WhiteHare]

I take it back.  With the latest coding, using SPI_CLOCK_DIV2 does work.  So, that should yield at least some improvement.

[perky]

If you don't use auto mode I'm not sure what will happen if a PacketSent happens and you then write data into the FIFO without clearing PacketSent first. The hope with the auto mode version is that when a PacketSent is detected it will exit from intermediate mode back to TX mode and restart a new transmission if there's something still in the FIFO, auto-clearing the PacketSent interrupt. No guarantee it will do that though.

Remember the FIFO empty condition happens when the last data has removed from the FIFO by the transmit logic, but it's actually been placed in the shift register and is therefore still transmitting, so FIFO empty happens at least 8 bits time before PacketSent. That timing could have strange effects, you might want to wait for PacketSent before sending the next packet (except the first one of course )

Mark.