Evasive flash.initialize() issues ...

[Humancell]

Hello,

I recently began to see some strange behaviors with some MoteinoMEGA boards that I've been working with.  I purchased these with the flash memory chip on them, and I use the 8-byte unique ID as an address for each board.

I've recently been connecting these to an external 5vdc power source, and I noticed that when connecting the wires there sometimes appears to be some "jitter" in the connection.  What I mean by this is that as I touch the wire to the terminal and then screw down the clamp the board will sometimes boot, and then reboot, and maybe even reboot a third time.  All of this occurs very quickly, and I completely understand this is due to the wire not being instantly secured.

The real issue that I started to see is that sometimes the flash chip will fail on flash.initialize().  When this occurs, obviously I can't get the unique ID.

What is even stranger are the next two points:

• Sometimes the flash.initialize() will work ... but I get back a BAD/WRONG unique ID.
• I've created a work-around, but I can't understand exactly why it would work ... but it does!

My work-around is to use resetUsingWatchdog() to reset the board in either situation - flash.initialize() failure or bad/wrong ID ... and this SOLVES either issue.

I created this solution as I noticed that when either failure occurred I could use the reset pin and everything would then work perfectly.

I can recreate this even using my FTDI connected serial interface.  If I push it on at an angle, slowly, and allow for a little intermittent contact I'll get flash.initialize() to fail ... and calling resetUsingWatchdog() will reset the board and it will then come up just fine.

Similarly, sometimes flash.initialize() will be successful, but the unique ID read from the chip will NOT be correct.  I know this as I display the unique ID every tiem on boot, and have logged the "correct" ID.  Again, in these instances if I call resetUsingWatchdog() it will reset the board and then boot up just fine ... and return the correct unique ID.

My real question is:  Why would resetUsingWatchdog() correct this situation?

I was originally thinking this was the flash chip somehow coming up in some weird mode, but I can't see how resetUsingWatchdog() would reset the memory chip ... so somehow it seems that maybe flash.initialize() is not setting up the pins correctly ... or??

Snippet of my current setup code:

Code: [Select]

  Serial.println(F("Attempting to initialize SPI Flash MEM ..."));    

  // TODO - need to understand why this is required when powering up!
  while(!flash.initialize()) {
    Serial.println(F("SPI Flash MEM failed to Initialize ... rebooting!"));    
    resetUsingWatchdog(TRUE);
  }

  Serial.print(F("SPI Flash Init OK. Unique MAC = ["));
  flash.readUniqueId();
  for (byte i=0; i < 8; i++)
  {
    GUIDbytes[i] = flash.UNIQUEID[i];
    if (GUIDbytes[i] < 0x10) Serial.print(F("0"));
    Serial.print(flash.UNIQUEID[i], HEX);
    if (i < 7) Serial.print(F(":"));
  }
  Serial.println(F("]"));

  // TODO - need to fix this! Checking for "bad" GUID
  if (GUIDbytes[0] != 0xD7 && GUIDbytes[1] != 0x68) {
    Serial.println(F("Suspect Unique ID detected ... did not start with D7:68 ... rebooting!"));    
    resetUsingWatchdog(TRUE);
  }

  GUIDstr = arrayToHexString(flash.UNIQUEID, 8);
  GUIDstr.toCharArray(GUIDchar,16);

  Serial.print(F("GUID = ["));
  Serial.print(GUIDstr);
  Serial.println(F("]"));

An example of what I see *sometimes* when the flash.initialize() works ...

Code: [Select]

Attempting to initialize SPI Flash MEM ...
SPI Flash Init OK. Unique MAC = [2B:20:08:02:0A:FF:00:15]
Suspect Unique ID detected ... did not start with D7:68 ... rebooting!
resetUsingWatchdog called: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~⸮
Attempting to initialize SPI Flash MEM ...
SPI Flash Init OK. Unique MAC = [D7:68:64:46:8F:2F:48:25]
GUID = [D76864468F2F4825]

I *am* looking for ways to filter these power transients, but am really interested in how resetUsingWatchdog() might be correcting this situation.

Thoughts?

[TomWS]

Interesting observations and creative solution.  I doubt you'll get a satisfying explanation for your odd behavior.  My theory is that each device has its own power on reset circuit that all work reliably if the power is applied cleanly, ie, no multiple edges and a fast rise time.  In cases where the power supply application is not clean, some devices come up cleanly, some don't and require certain code sequencing to get them out of their 'funk'.  Your wdt reset causes a resequencing after the power has stabilized and therefore produces clean results.

The processor should always power up cleanly if the BOD circuit is enabled.  The WDT code then runs when necessary.  This is probably the best assurance of a reliable power up/restart short of adding a HW WDT (which I use on all of my own mote designs via TPL5010 circuit).

[Humancell]

I appreciate your thoughts ... and agree with your suggestions.

One specific question:

If the resetUsingWatchdog() is working (which calls:  wdt_enable(WDTO_15MS); ) then it seems that it is the processor that is somehow in a bad state?

I can't think of any way that the call to resetUsingWatchdog() is resetting the flash chip ... or is there?

[TomWS]

I can't think of any way that the call to resetUsingWatchdog() is resetting the flash chip ... or is there?

No, not directly, but, again my theory, once the power supply has settled, then the flash initialization sequence effectively readies it for access.  Portions of the initialization sequence could be missed if the PS isn't stable yet.

One experiment you could try, if you can reliably reproduce a bad startup is simply put a reasonably long delay (a few hundred milliseconds) at the very beginning of setup().  This ensures that the power supply has settled before you're accessing your peripherals and might add evidence to the theory...