Author Topic: Sync bytes used by RFM69 library  (Read 5293 times)

WhiteHare

  • Hero Member
  • *****
  • Posts: 1300
  • Country: us
Sync bytes used by RFM69 library
« on: March 12, 2017, 11:31:58 AM »
I had always assumed that the RFM69 used a single sync byte, namely the network id.  However, it turns out that it uses TWO sync bytes.

The first is 45, and the second is 100.  Well, the 100 is clearly the netword ID.  But what is the 45 all about?

Does it use two bytes purely because the DS refers to it as a sync word, not a sync byte?  The DS does seem to imply that a single sync byte could also be used.

You can see for yourself by running this code:
Code: [Select]
  Serial.print(F("RegSyncValue1="));
  Serial.println(radio.readReg(REG_SYNCVALUE1));

  Serial.print(F("RegSyncValue2="));
  Serial.println(radio.readReg(REG_SYNCVALUE2));
after initializing the radio.
« Last Edit: March 12, 2017, 11:40:21 AM by WhiteHare »

perky

  • Hero Member
  • *****
  • Posts: 873
  • Country: gb
Re: Sync bytes used by RFM69 library
« Reply #1 on: March 12, 2017, 11:47:05 AM »
To maintain compatibility with the RFM12B that uses 0x2D plus sync word.
Mark.

WhiteHare

  • Hero Member
  • *****
  • Posts: 1300
  • Country: us
Re: Sync bytes used by RFM69 library
« Reply #2 on: March 12, 2017, 01:53:48 PM »
Thanks!  In my case I don't have any RFM12b's, so the extra syc byte is possibly wasted.

Felix

  • Administrator
  • Hero Member
  • *****
  • Posts: 6402
  • Country: us
    • LowPowerLab
Re: Sync bytes used by RFM69 library
« Reply #3 on: March 12, 2017, 04:47:02 PM »
Yes very much so, back when I created this library I thought there might be a time when we really care about RFM12B compatibility and possibly bridging etc, but ... it never materialized so that byte got lost in translation. Now it's a little tricky to remove it because everyone has had that byte in their sketches and deployed motes. I dont see a problem removing it though.

joelucid

  • Hero Member
  • *****
  • Posts: 867
Re: Sync bytes used by RFM69 library
« Reply #4 on: March 13, 2017, 01:48:07 AM »
The problem with short sync ids is that they lead to frequently mistaking noise for actual packets if the rssi threshold is set below the noise floor (which it is in the default rfm69 settings). You don't notice that directly since the library switches the radio to automatically discard packets with invalid crc. What you'll see instead is a short interruption in ability to rx every time a phantom packet hits.

With higher bitrates this is already noticeable with 2 byte sync. With one byte it would happen all the time.

To further elaborate: sync word detection is implemented as a bit shift register. So every bit gives you a new word that could match. With a one byte sync id you should get a phantom packet every 256 bits, or every 32 bytes, or every 5ms at the 55khz default settings on average. At 2 bytes it's about one per second.

This is why I'm using 3 bytes which averages one phantom packet in around 5 minutes at 55khz.

« Last Edit: March 13, 2017, 08:55:59 AM by joelucid »

Felix

  • Administrator
  • Hero Member
  • *****
  • Posts: 6402
  • Country: us
    • LowPowerLab
Re: Sync bytes used by RFM69 library
« Reply #5 on: March 13, 2017, 08:54:57 AM »
Very good points, thanks joe!

WhiteHare

  • Hero Member
  • *****
  • Posts: 1300
  • Country: us
Re: Sync bytes used by RFM69 library
« Reply #6 on: March 13, 2017, 10:27:04 AM »
The problem with short sync ids is that they lead to frequently mistaking noise for actual packets if the rssi threshold is set below the noise floor (which it is in the default rfm69 settings). You don't notice that directly since the library switches the radio to automatically discard packets with invalid crc. What you'll see instead is a short interruption in ability to rx every time a phantom packet hits.

With higher bitrates this is already noticeable with 2 byte sync. With one byte it would happen all the time.

To further elaborate: sync word detection is implemented as a bit shift register. So every bit gives you a new word that could match. With a one byte sync id you should get a phantom packet every 256 bits, or every 32 bytes, or every 5ms at the 55khz default settings on average. At 2 bytes it's about one per second.

This is why I'm using 3 bytes which averages one phantom packet in around 5 minutes at 55khz.

IF  a preamble is being used (as it now seems likely),  does that cut down at all on the frequency of occurance of the phantom packets?  i.e. does the phantom packet have to, in some sense, have to be preceeded by a well formed preamble?  i.e. does having a long preamble also act as a filter of sorts, or not?

joelucid

  • Hero Member
  • *****
  • Posts: 867
Re: Sync bytes used by RFM69 library
« Reply #7 on: March 13, 2017, 12:01:59 PM »
Quote
IF  a preamble is being used (as it now seems likely),  does that cut down at all on the frequency of occurance of the phantom packets?  i.e. does the phantom packet have to, in some sense, have to be preceeded by a well formed preamble?  i.e. does having a long preamble also act as a filter of sorts, or not?

I don't think so - I'm seeing way too many phantom packets for that to be true. Once the RSSI thresh is triggered the radio assumes what it sees is preamble and does AGC/AFC on whatever it sees. Once RX is ready it just streams the signal through the Sync Id bitshift logic until it sees a match.

One way to prevent this is to set an appropriate RSSI threshold above the noise floor. That sounds easy but is pretty hard in the general case. So for me RSSI thresh at 255 and 3 byte sync id it is.

Joe

perky

  • Hero Member
  • *****
  • Posts: 873
  • Country: gb
Re: Sync bytes used by RFM69 library
« Reply #8 on: March 13, 2017, 01:37:23 PM »
In order to decode properly the logic has to settle to a point where it can actually reliably decode, so this has to happen before the preamble can be decoded. So I think Joe is right, the assumption by the logic will be that there is a preamble once the RSSI threshold has been met.

However I think there is a delimiter at the end of a preamble that indicates the start of a sync word, so once it starts decoding the preamble it would be looking for two consecutive bits being the same before starting the sync word shifting process. So when you get noise that two consecutive condition is likely to occur very often, hence multiple triggers. I would expect it to start the decoding process after a RSSI interrupt, but not start another one until the RSSI interrupt has been cleared somehow, e.g. exiting RX mode and re-entering it again or restarting. If that is actually what happens it means the sync word can't be continually shifting and comparing.

Mark.

perky

  • Hero Member
  • *****
  • Posts: 873
  • Country: gb
Re: Sync bytes used by RFM69 library
« Reply #9 on: March 14, 2017, 11:54:04 AM »
[Edit: this post has been moved from the other thread about minimum packet sizes]

I'm still not clear on exactly how the sync word detection works. You can have zero bytes of sync word even in packet mode, which would imply some kind of delimiter after the preamble (otherwise how does it byte align, or know where the data actually starts?). What happens though when it detects that delimiter and starts the sync word sampling?

I suspect it does what the EZ-Radio does (see links below), which is to shift the next n bits as determined by the sync word length register, and then compare. If the answer is no match, start to detect a preamble again, and once it thinks it's got that look again for the delimiter and repeat. This would give the impression of continual scanning.

One consequence of this might be that the number of preamble bytes compared to the number of sync word bytes might be important to be able to capture every packet. Imagine if you had 2 preamble bytes and 8 sync word bytes. In that case the logic might start to sample sync words on noise, but if a real packet then started the preamble for that may finish before the original sampling process had finished, and you would lose that new packet. If, however, the preamble was 8 bytes and the sync word 2 bytes then it will have guaranteed to finish sampling the sync word, recognize the new preamble, then get the delimiter at the end and all is well. Joe's code may well be working because the preamble is larger than the sync word...

BTW the delimter in the EZ-Radio is simply delaying the bit stream by one bit and XORing the result. It doesn't say how many bits this covers, but it's analogous to the repeating the last bit in the preamble if it were just one bit.

http://community.silabs.com/t5/Proprietary-Knowledge-Base/EZRadioPRO-and-Sync-Word-Detection/ta-p/113031
http://community.silabs.com/t5/Proprietary-Knowledge-Base/EZRadioPRO-and-Sync-Word-Selection/ta-p/113455
http://community.silabs.com/t5/Proprietary-Knowledge-Base/Understanding-the-receive-state-machine-on-EZRadioPro-and/ta-p/151733

Mark.
« Last Edit: March 14, 2017, 11:55:41 AM by perky »

joelucid

  • Hero Member
  • *****
  • Posts: 867
Re: Sync bytes used by RFM69 library
« Reply #10 on: March 14, 2017, 03:13:40 PM »
Quote
I'm still not clear on exactly how the sync word detection works. You can have zero bytes of sync word even in packet mode, which would imply some kind of delimiter after the preamble (otherwise how does it byte align, or know where the data actually starts?).

There may be a delimiter, I don't know. It'd be pretty easy to check: set a one byte sync word with the same pattern as the preamble and check if you get ADDRESSMATCH if the other side is merely sending extended preamble.

Quote
I suspect it does what the EZ-Radio does (see links below), which is to shift the next n bits as determined by the sync word length register, and then compare. If the answer is no match, start to detect a preamble again, and once it thinks it's got that look again for the delimiter and repeat. This would give the impression of continual scanning.

That is definitely not what's happening. Once the radio starts looking for the sync word it does so until it finds it or you change mode or issue RXRESTART. The entire approach of running at an RSSI thresh of 255 is predicated on this.

Joe

perky

  • Hero Member
  • *****
  • Posts: 873
  • Country: gb
Re: Sync bytes used by RFM69 library
« Reply #11 on: March 14, 2017, 05:42:21 PM »
Quote
There may be a delimiter, I don't know. It'd be pretty easy to check: set a one byte sync word with the same pattern as the preamble and check if you get ADDRESSMATCH if the other side is merely sending extended preamble.

That's a worthwhile experiment to do. I'll see if I can knock up a test.

Quote
I suspect it does what the EZ-Radio does (see links below), which is to shift the next n bits as determined by the sync word length register, and then compare. If the answer is no match, start to detect a preamble again, and once it thinks it's got that look again for the delimiter and repeat. This would give the impression of continual scanning.
Quote
That is definitely not what's happening. Once the radio starts looking for the sync word it does so until it finds it or you change mode or issue RXRESTART. The entire approach of running at an RSSI thresh of 255 is predicated on this.

I'm not suggesting the user restarts the preamble detection, I'm saying the radio itself might automatically. So the receive process might be:

1) go into RX mode
2) radio hunts for the preamble, once detected then goto 3
3) radio waits for the delimiter (bit repeat?), once detected then goto 4
4) radio bit shifts the next n bits and only at the end compares. If match goto 5, else go back to 2.
5) receive packet.

So that would scan indefinitely. And if the preamble is longer than the sync word you won't lose any packets with RSSI threshold set at 255, but if the sync word is longer than the preamble you might.

That's the way the EZRadioPRO appears to do it, and it therefore would allow arbitrary sync word patterns, although I note it recommends sync words with 'good auto-correlation functions' and has a warning about 0xAA or 0x55 patterns but that might be for the non-PRO versions.

So are we really sure this isn't what is happening? It may well just do a simple scan as per the non-PRO EZRadios, but I can see the above potentially giving the same test results if the number of preamble bytes is more than the sync word.

Mark.


WhiteHare

  • Hero Member
  • *****
  • Posts: 1300
  • Country: us
Re: Sync bytes used by RFM69 library
« Reply #12 on: March 14, 2017, 06:09:46 PM »
It sounds like it would be easy to test your theory, Perky.  Use, say, 8 sync bytes on the Tx and only 2 on the Rx, but embed the 2 two sync bytes that the Rx expects not at the beginning but somewhere else (I guess as the last 2 of the 8 tx'd sync bytes).  If Joe's right, then the receiver should catch the packet.  If you're right, it won't.

perky

  • Hero Member
  • *****
  • Posts: 873
  • Country: gb
Re: Sync bytes used by RFM69 library
« Reply #13 on: March 14, 2017, 06:25:08 PM »
Yeah, but I'm leaning towards Joe's interpretation now, notwithstanding the description in the sx1231h datasheet does say continual ;) It's a pity Samtech are so silent on stuff like this compared to SiLabs.

Mark.
« Last Edit: March 14, 2017, 09:23:05 PM by perky »

joelucid

  • Hero Member
  • *****
  • Posts: 867
Re: Sync bytes used by RFM69 library
« Reply #14 on: March 14, 2017, 07:10:52 PM »
During preamble reception the radio does AGC, AFC etc. I don't think it's reasonable to assume that it can detect whether the signal actually is preamble at the same time. Also the frequency of phantom packets would be much lower than what I'm seeing if a full preamble had to be seen to restart sync address detection.

perky

  • Hero Member
  • *****
  • Posts: 873
  • Country: gb
Re: Sync bytes used by RFM69 library
« Reply #15 on: March 15, 2017, 06:46:00 AM »
During preamble reception the radio does AGC, AFC etc. I don't think it's reasonable to assume that it can detect whether the signal actually is preamble at the same time.

AGC and AFC can either be triggered manually or by a restart using RSSI threshold, so no AFC or AGC could be used when continually scanning with RSSI threshold set to 255, but that's the same situation as yours I think.

Also the frequency of phantom packets would be much lower than what I'm seeing if a full preamble had to be seen to restart sync address detection.

Full preamble length is not know by the receiver (at least the length of a properly decoded preamble it actually receives  isn't), so the preamble length that is detected is really relatively short in comparison to the full preamble length, what's important is waiting for the delimiter at the end of that preamble. So this means more chance of random 1010 type patterns retrigerring the sync word detection mechanism than if using full preamble length.

Mark.

joelucid

  • Hero Member
  • *****
  • Posts: 867
Re: Sync bytes used by RFM69 library
« Reply #16 on: March 15, 2017, 07:17:59 AM »
Quote
waiting for the delimiter at the end of that preamble

Sounds like your theory has morphed into the radio just adding a delimiter bit in front of the sync word when comparing the bitshift register to avoid detecting address match during preamble. That's certainly possible and still compatible with the phantom packet frequency I observe. If its a 4 bits delimiter I think it's probably incompatible with my observations.

That wouldn't change the conclusion that even a 2 byte sync word is cutting it close when running at a 255 thresh.

Joe

perky

  • Hero Member
  • *****
  • Posts: 873
  • Country: gb
Re: Sync bytes used by RFM69 library
« Reply #17 on: March 15, 2017, 07:50:27 AM »
Yes, I suspect this is in reality is really just a continually shifting register as you stated but with a short delimiter, possible only one repeated bit, at the end of the preamble. I can't see how using zero sync words could actually work without it. In which case chosing sync words with 'good auto-correlation functions' would reduce the false triggering:
http://community.silabs.com/t5/Proprietary-Knowledge-Base/EZRadioPRO-and-Sync-Word-Selection/ta-p/113455

Mark.

joelucid

  • Hero Member
  • *****
  • Posts: 867
Re: Sync bytes used by RFM69 library
« Reply #18 on: March 15, 2017, 08:05:23 AM »
You know, now that I think about it I think I have evidence for the delimiter bit: I have some code that controls these 433mhz rf AC-sockets. I do it in packet mode, with 0 byte preamble, and send the bytes that generate the reverse engineered code. That's always worked well for me.

But when Tom tested it, his sockets wouldn't respond. When he looked at the generated signal he consistently found an extra 1 transmitted in front of the sequence. In the end he just added a couple of 0's to the actual sequence to separate it in time from the control code and that worked. But it seems that this 1 might have been the delimiter.

Joe

WhiteHare

  • Hero Member
  • *****
  • Posts: 1300
  • Country: us
Re: Sync bytes used by RFM69 library
« Reply #19 on: March 15, 2017, 08:27:46 AM »
Yes, I suspect this is in reality is really just a continually shifting register as you stated but with a short delimiter, possible only one repeated bit, at the end of the preamble. I can't see how using zero sync words could actually work without it. In which case chosing sync words with 'good auto-correlation functions' would reduce the false triggering:
http://community.silabs.com/t5/Proprietary-Knowledge-Base/EZRadioPRO-and-Sync-Word-Selection/ta-p/113455

Mark.

I'm assuming (?) that the data whitening applies to the payload only, and not the sync words. 

Is the property of "good auto-correlation" an abstract mathematical property that's hardware independent, or rather does it  (I'm guessing) depend on how the underlying hardware works?  In which case, how do we choose good sync words for the RFM69 that have good "auto correlation"?  I have not even an inkling, other than to apparently avoid repeating bit patterns (as suggested by the Silicon Labs link) that look like a preamble.

Ay yi yi.  Sometimes it feels like we practically have to be able to build an RFM69 all the way up from gates and transistors in order just to use one!
« Last Edit: March 15, 2017, 08:48:46 AM by WhiteHare »

perky

  • Hero Member
  • *****
  • Posts: 873
  • Country: gb
Re: Sync bytes used by RFM69 library
« Reply #20 on: March 15, 2017, 08:52:38 AM »
@ joelucid: Aha, that's an interesting observation.

BTW my system uses 5 preambles and 3 sync bytes, and I set the sync word to 0xAAAAAA to see what happens. As you predicted, it didn't work. Setting it to 0xAAAAAB was fine.

I then repeated the experiment with 0x555555. That didn't work either. So even with a delimiter the sync word shift register is shifting data even during the preamble, the output of the sync word comparitor probably gets enabled once the delimeter is detected but it already has the preamble shifted into it so instantly matches. I think the difference between the 0xAA and 0x55 cases is it requires one extra bit to be shifted in to generate a match.

So I think that answers the question about the shifting.

Mark.

joelucid

  • Hero Member
  • *****
  • Posts: 867
Re: Sync bytes used by RFM69 library
« Reply #21 on: March 15, 2017, 09:10:37 AM »
Quote
So I think that answers the question about the shifting.

Interesting! Thanks for sharing.

perky

  • Hero Member
  • *****
  • Posts: 873
  • Country: gb
Re: Sync bytes used by RFM69 library
« Reply #22 on: March 15, 2017, 09:49:29 AM »
I'm assuming (?) that the data whitening applies to the payload only, and not the sync words. 

Is the property of "good auto-correlation" an abstract mathematical property that's hardware independent, or rather does it  (I'm guessing) depend on how the underlying hardware works?  In which case, how do we choose good sync words for the RFM69 that have good "auto correlation"?  I have not even an inkling, other than to apparently avoid repeating bit patterns (as suggested by the Silicon Labs link) that look like a preamble.

Ay yi yi.  Sometimes it feels like we practically have to be able to build an RFM69 all the way up from gates and transistors in order just to use one!

I would guess auto-correlation is a measure of how well bit shifted versions of the same value match, but I'm as in the dark as you are as to how to calculate such a thing other than as you say avoiding patterns that look like a preamble.

I did find this post that might shed a little light on it:
https://e2e.ti.com/support/rf__digital_radio/etc_rf/f/228/t/313209

Mark.

WhiteHare

  • Hero Member
  • *****
  • Posts: 1300
  • Country: us
Re: Sync bytes used by RFM69 library
« Reply #23 on: March 15, 2017, 01:56:59 PM »
I suppose it might be telling that Semtech's default for the number of sync bytes (following a power-up or reset of the sx1231h) is 4, according to the DS.  That's consistent with Joe's theory that 3 is barely the minimum.

perky

  • Hero Member
  • *****
  • Posts: 873
  • Country: gb
Re: Sync bytes used by RFM69 library
« Reply #24 on: March 15, 2017, 03:31:36 PM »
I think thats the preamble bare minimum for 300kHz BR.
Mark.

WhiteHare

  • Hero Member
  • *****
  • Posts: 1300
  • Country: us
Re: Sync bytes used by RFM69 library
« Reply #25 on: March 15, 2017, 05:03:52 PM »
Which number?  3 bytes or 4 bytes?


joelucid

  • Hero Member
  • *****
  • Posts: 867
Re: Sync bytes used by RFM69 library
« Reply #27 on: March 16, 2017, 03:39:56 AM »
Quote
but I'm as in the dark as you are as to how to calculate such a thing other than as you say avoiding patterns that look like a preamble.

The auto-correlation at n is just the sum of the product of the de-dc biased sync word with itself, shifted by n bits. It obviously has a spike at n = 0. It might have other lobes at higher or lower n. The extent of these non 0 lobes shows how much it repeats.

Choosing a good sync word seems to be a science in itself. Here's how bluetooth does it: https://books.google.de/books?id=mUyiREePWHwC&pg=PT92&lpg=PT92&dq=sync+word+autocorrelation&source=bl&ots=0AvJU8ApVs&sig=mqqPrelMnGJ2WxF6xt7xlF7iMo8&hl=en&sa=X&redir_esc=y#v=onepage&q=sync%20word%20autocorrelation&f=false.

And if you use the best two byte sync word (using the third byte as your network address) I bet the side effect is that you might be using the same one as all the others :).

For me this goes into the "later" stack.

EDIT:

I would however caution everybody to switch off auto-discard of non CRC matched packets from time to time and see how much junk hits. My guess is you'll be surprised, esp. if you use the RFM69 default settings. If you're seeing 1% failed transmissions on an otherwise perfect link a too short sync-word might well be the explanation.

Joe
« Last Edit: March 16, 2017, 03:55:35 AM by joelucid »