sendWithRetry(): Missed ACKs

[Neko]

How do you handle it when sendWithRetry() misses the ACK and sends a message twice? In my application, a node is sending event counts. When the gateway sends an ACK and the node misses the ACK, it sends again, and the events get counted twice.

I have two ways of checking for this. One is to give each message a unique ID so that the GW can detect and reject duplicates. The other is to discount messages that arrive within 40 ms of one another.

But these seem like hacks. Is there a better way, either to cut down on missed ACK's or deal with them?

[Felix]

Similar to one of your suggestions, in the PiGateway project, repeated packages from the same node, received within less than 500ms, are discarded. It is assumed a node will never report that often. Ultimately data more granular is not normally useful in such an application.

This is a simple check that is achieved using this piece of code from gateway.js:

Code Select Expand

      //check for duplicate messages - this can happen when the remote node sends an ACK-ed message but does not get the ACK so it resends same message repeatedly until it receives an ACK
      if (existingNode.updated != undefined && (Date.now() - existingNode.updated < 500) && msgHistory[id] == msgTokens)
      {
        console.log("   DUPLICATE, skipping...");
        return;
      }

The root of the problem is much harder to solve because there are multiple ways and reasons that any node can miss an ACK.
One potential way to reduce missed ACKs is to randomize their inter-send delays (and not used a fixed 20/30ms delay). But this is a bit elusive.
A more systematic approach is to watch the packets go in/out and determine the best sequence of timings between the real packet and the ACK. That would reduce the chance that a delayed ACK meets collisions and other interference since we're putting it back to back to a transmission (+ an actual expected delay incurred by the receiver who needs to process the real packet before sending it's ACK). It gets a little complicated since you have to either use a logic analyzer to watch the SPI traffic and determine which state the radio is in, or much better - to use a tool such as CurrentRanger which will show the current patterns of the node when the radio is turned on, ramps up, goes into TX mode and transmits, turns off, goes into RX mode to listen for the ACK, etc, node sleeps. Here's an example of a packet capture with the CR:



I did it a few times for debugging and to find some optimal high speed transmission timings, but it can take days to fiddle with this to count timings and extract maximum performance.
I wish I had the time to go deep into this stuff and write a blog about it. It must be a less than great thing to say that it's much easier to solve this in software than spend all that time trying to chase every packet to ensure it is delivered properly. And there are still chances for collisions, interference, brownouts, who knows what else, that messes up a node's receipt of an ACK.