Author Topic: Help understanding, no ACK and missing every other message  (Read 1471 times)

magnet_man

  • NewMember
  • *
  • Posts: 2
Help understanding, no ACK and missing every other message
« on: October 15, 2019, 04:41:23 PM »
Hello, I ran into two problems getting started with a simple node/gateway setup. As I was typing up a post to ask for help, I ended up doing some side research and have successfully made the problems go away (yay!). However, I don't understand why my solutions work (boo!). So I can increase complexity of the design, I'm hoping for some explanations or help finding the documentation that shows why solutions work.

I got 2x Moteino's (RFM69HCW with 433 MHz and no flash) to test out and hopefully use to begin doing home automation with. I worked my way up from simple blink's to transmitting random data from one to the other and was successful. Got ACKs everything. I then installed the pre-made Pi ino file from GitHub (https://github.com/LowPowerLab/RFM69/blob/master/Examples/PiGateway/PiGateway.ino) on one of the Moteinos and hooked it up to my Pi3 (no mightyhat or additional hardware). I installed and configured the gateway package on the Pi. Finally, I created a simple weather node moteino on the other one by hooking up a Si7021 T/RH sensor.

Here's where the problems cropped up.
I had successfully sent messages from the weather node (both T and RH) to the Pi and had seen appropriate responses on the Gateway web interface (auto populated node, pulled START, RSSI, T, and RH data). The two issues that arose and their subsequent solution are listed below.

Issue 1) With the weather node connected to my laptop and monitoring the serial stream, the node claims that no ACK was received (using ATC and sendWithRetry) even though the data was successfully updating on the Gateway web interface.

Solution: Increase the retryWaitTime to 250 ms. This gave a 'no acknowledgment' rate of about 1:200 messages sent. Reducing the wait increased the number of sent messages that did not receive acknowledgement.

Question: Is this wait time a function of message length? If so is there a rule of thumb for how long to wait for a given message size (in this case 15 characters)?

Issue 2) Every other message was lost. I set the weather node to transmit once every 15 seconds, then the Gateway only updated every 30 seconds. This behavior extended to the Start message as in the gateway updated the start timer, the first weather node data was skipped, but the second message was received. Removing the start message meant that the first weather message updated the gateway and the second message was skipped.

Solution: After comparing my node code to the RandomNumber example code the only major difference was that my code did not check for incoming messages and was missing a radio.receiveDone(); command prior to the broadcast command.

Question: Why did inserting this code fix the errant behavior? 

Here is my Gateway config. All other parts of the gateway ino file are the same as the Github's (downloaded in the past 2 days):
Code: [Select]
#define NODEID          1 //the ID of this node
#define NETWORKID     87 //the network ID of all nodes this node listens/talks to
#define FREQUENCY     RF69_433MHZ //Match this with the version of your Moteino! (others: RF69_433MHZ, RF69_868MHZ)
#define ENCRYPTKEY    "a16longencryptke" //identical 16 characters/bytes on all nodes, not more not less!
#define IS_RFM69HW_HCW  //uncomment only for RFM69HW/HCW! Leave out if you have RFM69W/CW!
#define ACK_TIME       30  // # of ms to wait for an ack packet

#define ENABLE_ATC    //comment out this line to disable AUTO TRANSMISSION CONTROL
#define ATC_RSSI      -75  //target RSSI for RFM69_ATC (recommended > -80)

// Serial baud rate must match your Pi/host computer serial port baud rate!
#define SERIAL_EN     //comment out if you don't want any serial verbose output
#define SERIAL_BAUD  19200

This is my weather node moteino code. Code pulled from the Node example provided with the Arduino library install (removed licensing comments for post brevity) and mixed with some existing Si7021 code I already wrote. I know my way of generating the characters for the message is unorthodox, but I tested it out by using the sprintf statements and I got the same results.

Code: [Select]
// Sample RFM69 sender/node sketch, with ACK and optional encryption, and Automatic Transmission Control
// **********************************************************************************
#include <RFM69.h>         //get it here: https://www.github.com/lowpowerlab/rfm69
#include <RFM69_ATC.h>     //get it here: https://www.github.com/lowpowerlab/rfm69
#include <SPIFlash.h>      //get it here: https://www.github.com/lowpowerlab/spiflash
#include <SPI.h>           //included with Arduino IDE install (www.arduino.cc)
#include "SparkFun_Si7021_Breakout_Library.h"
#include <Wire.h>

//*********************************************************************************************
//************ IMPORTANT SETTINGS - YOU MUST CHANGE/CONFIGURE TO FIT YOUR HARDWARE ************
//*********************************************************************************************
#define NODEID        2    //must be unique for each node on same network (range up to 254, 255 is used for broadcast)
#define NETWORKID     87  //the same on all nodes that talk to each other (range up to 255)
#define GATEWAYID     1
//Match frequency to the hardware version of the radio on your Moteino (uncomment one):
#define FREQUENCY   RF69_433MHZ
#define ENCRYPTKEY    "a16longencryptke" //exactly the same 16 characters/bytes on all nodes!
#define IS_RFM69HW_HCW  //uncomment only for RFM69HW/HCW! Leave out if you have RFM69W/CW!
//*********************************************************************************************
//Auto Transmission Control - dials down transmit power to save battery
//Usually you do not need to always transmit at max output power
//By reducing TX power even a little you save a significant amount of battery power
//This setting enables this gateway to work with remote nodes that have ATC enabled to
//dial their power down to only the required level (ATC_RSSI)
#define ENABLE_ATC    //comment out this line to disable AUTO TRANSMISSION CONTROL
#define ATC_RSSI      -80
//*********************************************************************************************
#define SERIAL_BAUD   115200
#define LED           9 // Moteinos have LEDs on D9
Weather sensor;
 

#if defined (MOTEINO_M0) && defined(SERIAL_PORT_USBVIRTUAL)
  #define Serial SERIAL_PORT_USBVIRTUAL // Required for Serial on Zero based boards
#endif


#ifdef ENABLE_ATC
  RFM69_ATC radio;
#else
  RFM69 radio;
#endif

static float humidity = 0;
static float tempf = 0;

void setup() {
  Serial.begin(SERIAL_BAUD);
  radio.initialize(FREQUENCY,NODEID,NETWORKID);
#ifdef IS_RFM69HW_HCW
  radio.setHighPower(); //must include this only for RFM69HW/HCW!
#endif
  radio.encrypt(ENCRYPTKEY);

//Auto Transmission Control - dials down transmit power to save battery (-100 is the noise floor, -90 is still pretty good)
//For indoor nodes that are pretty static and at pretty stable temperatures (like a MotionMote) -90dBm is quite safe
//For more variable nodes that can expect to move or experience larger temp drifts a lower margin like -70 to -80 would probably be better
//Always test your ATC mote in the edge cases in your own environment to ensure ATC will perform as you expect
#ifdef ENABLE_ATC
  radio.enableAutoPower(ATC_RSSI);
#endif

  char buff[50];
  sprintf(buff, "\nTransmitting at %d Mhz...", FREQUENCY==RF69_433MHZ ? 433 : FREQUENCY==RF69_868MHZ ? 868 : 915);
  Serial.println(buff);


#ifdef ENABLE_ATC
  Serial.println("RFM69_ATC Enabled (Auto Transmission Control)\n");
#endif

  // Set LED to output and turn off in preparation for blink
  pinMode(LED, OUTPUT);
  digitalWrite(LED,LOW);
  // Activate the Si7021 sensor
  sensor.begin();
  // Tell Gateway the node is alive
  radio.sendWithRetry(GATEWAYID, "START", 5);
}

// Timing variables
static long blinkRate = 1000;
static unsigned long lastBlink = 0;
static long sensorRate = 15000;
static unsigned long lastSensor = 0;
static unsigned long currentms = 0;

// Current state of the LED
bool ledState = false;

// Character buffers
static char TBuff[18];
static char dataChar[8];

void loop() {
  currentms = millis();
 
  // Blink LED
  if (currentms - lastBlink >= blinkRate)
  {
    lastBlink = currentms;
    ledState = !ledState;
    digitalWrite(LED,ledState);
  }

  // Get sensor data and transmit to gateway
  if (currentms - lastSensor >= sensorRate)
  {
    lastSensor = currentms;
   
    // Get Sensor data, convert C to F
    humidity = sensor.getRH();
    tempf = sensor.getTemp();
    tempf = (1.8*tempf)+32;
    Serial.print("Temp:");
    Serial.print(tempf);
    Serial.print("F, ");

    Serial.print("Humidity:");
    Serial.print(humidity);
    Serial.println("%");

    // Convert temperature to char array
    dtostrf(tempf, 8, 2, dataChar);
    int c = 2;
    // Prep message in TBuff
    // Format will be as follows
    //  F:nnn.nn H:nn.nn   <---- Message format
    //  0123456789012345

    TBuff[0] = 'F';
    TBuff[1] = ':';
    // Copy the chars out of dataChar into TBuff ignoring white space
    for(int i = 0;i < 8;i++)
    {
      if(dataChar[i] != ' ')
        {TBuff[c++] = dataChar[i];}
    }
    TBuff[7] = ' ';
    TBuff[8] = ' ';
    TBuff[9] = 'H';
    TBuff[10] = ':';
    c = 11;

    // Repeat steps for humidity
    dtostrf(humidity,8,2,dataChar);
    for(int i = 0;i < 8;i++)
    {
      if(dataChar[i] != ' ')
        {TBuff[c++] = dataChar[i];}
    }

    // See what the array looks like before sending
    Serial.print("Send buffer >");Serial.print(TBuff);Serial.println("<");

    // Send buffer to gateway
    radio.receiveDone();
    if (radio.sendWithRetry(GATEWAYID, TBuff, 16))
      {Serial.println("data received");}
    else
      {Serial.println("data not received");}   
  }
 
}

Felix

  • Administrator
  • Hero Member
  • *****
  • Posts: 6866
  • Country: us
    • LowPowerLab
Re: Help understanding, no ACK and missing every other message
« Reply #1 on: October 16, 2019, 10:50:58 AM »
1) The retry wait time is not a function of packet length. At the default library settings a packet transmission (up to the max 60+ bytes) occurs in less than a few ms. The retry wait time is a magnitude larger to spread the packets out, and it would be best if this was randomized, but it was set to a fixed value for simplicity.

2) The receiveDone() is required to put the radio module in RX mode. Once a packet is received it leaves RX mode and waits for the user program to empty the received message buffers.
Hence that has to be done as soon as possible. Hence the radio cannot receive anything if it's in any other mode than RX.

PS. Unfortunately these "example code works, why doesn't my code work" questions are sometimes the most difficult to answer. There are other factors involved other than code, like how the modules are physically connected and their antennas positioned. When testing at close range this has lesser importance since even without an antenna a Moteino can reach across the desk, albeit with a poor RSSI.

magnet_man

  • NewMember
  • *
  • Posts: 2
Re: Help understanding, no ACK and missing every other message
« Reply #2 on: October 17, 2019, 11:02:20 AM »
Thank you Felix for the information. I do have some follow up clarifications

1) The retry wait time is not a function of packet length. At the default library settings a packet transmission (up to the max 60+ bytes) occurs in less than a few ms. The retry wait time is a magnitude larger to spread the packets out, and it would be best if this was randomized, but it was set to a fixed value for simplicity.

Is 250 ms typical value for retrywaittime? This stumped me for a while because I read somewhere that 40ish ms would be sufficient (can't remember where or what circumstances this value was set). When trying to find a suitable value, is trial and error the best way?

2) The receiveDone() is required to put the radio module in RX mode. Once a packet is received it leaves RX mode and waits for the user program to empty the received message buffers.
Hence that has to be done as soon as possible. Hence the radio cannot receive anything if it's in any other mode than RX.

This is more me piecing the information together and making sure I understand what's going on under the hood: So, since the sendwithretry() is waiting for an acknowledgement, and acknowledgements are messages with no payload, sendwithretry() put the node in listening mode which is why the receiveDone() is required in order to send the next message?

PS. Unfortunately these "example code works, why doesn't my code work" questions are sometimes the most difficult to answer. There are other factors involved other than code, like how the modules are physically connected and their antennas positioned. When testing at close range this has lesser importance since even without an antenna a Moteino can reach across the desk, albeit with a poor RSSI.

I can totally understand this and really do appreciate you taking the time to answer questions :)

Felix

  • Administrator
  • Hero Member
  • *****
  • Posts: 6866
  • Country: us
    • LowPowerLab
Re: Help understanding, no ACK and missing every other message
« Reply #3 on: October 17, 2019, 11:21:29 AM »
Is 250 ms typical value for retrywaittime? This stumped me for a while because I read somewhere that 40ish ms would be sufficient (can't remember where or what circumstances this value was set). When trying to find a suitable value, is trial and error the best way?

This is more me piecing the information together and making sure I understand what's going on under the hood: So, since the sendwithretry() is waiting for an acknowledgement, and acknowledgements are messages with no payload, sendwithretry() put the node in listening mode which is why the receiveDone() is required in order to send the next message?

I can totally understand this and really do appreciate you taking the time to answer questions :)
As mentioned before, a packet takes up to a few ms to completely transmit, even at maximum length. The radio module can then switch to RX and start receiving packets. This is done by calling receiveDone() which returns a boolean TRUE if a packet was received, or FALSE and/or puts the radio to RX mode if not already in RX.

The sendWithRetry() function just sends multiple packets (up to default or specified #, with default/specified delay between them) as long as no ACKs are received back.
The inter packet delay should be at least 10ms to give time to the receiver to parse the packet, determine an ACK is needed, and a few ms for the roundtrip of the ACK. In the examples it is typically given a generous 30-40ms. In most cases that is sufficient, and 250ms is way too long unless the receiver is busy doing other things (this is a bad practice, seen many times) after receiving a packet. Ie the receiver is coded to go on a coffee break after receiving a packet that has the ACK_REQUEST bit ON, instead of IMMEDIATELY replying back with an ACK. That is a common overlook that can cause these delays to be perceived as proportional to the length of the packet.

PS. To clarify, "ListenMode" is another operating mode of the radio module, distinct from the RX mode, which is still experimental and has been reported not to be fully reliable - this involves a few registers of the radio which quickly transition between RX mode and SLEEP, in order to reduce the consumption (rather than always be in RX mode).