Stop getting massages when lightning strikes nearby

[LukaQ]

This is now the second time I've seen this, I stop getting massages, gateway itself is still working as if RPi3, sw reboot doesn't fix that. I have to power off and remove power (or probably just reset Moteino). Gateway is indoor, PRi3 is powered with good power brick, which also power other Rpi2. Moteino is basic one (no flash), that is minimal for gateway

Has anyone else find that?

[TomWS]

It's tough to specifically attribute Moteino 'hangs' (or as I like to say, 'departures into parts unknown') to lightning, however, I do have a Watchdog timer on every 'Mote' I have just to recover from such an event (which do happen from time to time).  In my case I use hardware watchdogs (TPL5010) because they're easy and reliable (and consume virtually no power), but built-in WDT may be good enough for this application (and easier to add after the fact).

UPDATE: I just recall that I do have one device, implemented with a Moteino R4 / NodeMCU ESP8266 combination, where, when we get power spikes (due to lightning or wind blown trees 'playing' with the power lines) it will occasionally get lost and require power cycling to reset the Mote - the ESP8266  recovers nicely.  I didn't put in HW WD in this case and even failed to connect the Moteino Reset to the ESP8266 and I curse these decisions every time I have to go into the attic to cycle the power on the unit.  It was a 'quick and dirty' project 'thrown' together from spare parts.  One of these days...
BTW, this particular project uses the RFM69 as an OOK transmitter and doesn't even use receive...

Good luck!
Tom

[Felix]

I never observed anything like this that I can associate with power outages or lightning strikes etc.
I would recommend using WDT on any mission critical nodes like the Gateway.

[LukaQ]

I never observed anything like this that I can associate with power outages or lightning strikes etc.
I would recommend using WDT on any mission critical nodes like the Gateway.

And by that you mean hardware WDT like Tom said?

[LukaQ]

It's tough to specifically attribute Moteino 'hangs' (or as I like to say, 'departures into parts unknown') to lightning, however, I do have a Watchdog timer on every 'Mote' I have just to recover from such an event (which do happen from time to time).  In my case I use hardware watchdogs (TPL5010) because they're easy and reliable (and consume virtually no power), but built-in WDT may be good enough for this application (and easier to add after the fact).

UPDATE: I just recall that I do have one device, implemented with a Moteino R4 / NodeMCU ESP8266 combination, where, when we get power spikes (due to lightning or wind blown trees 'playing' with the power lines) it will occasionally get lost and require power cycling to reset the Mote - the ESP8266  recovers nicely.  I didn't put in HW WD in this case and even failed to connect the Moteino Reset to the ESP8266 and I curse these decisions every time I have to go into the attic to cycle the power on the unit.  It was a 'quick and dirty' project 'thrown' together from spare parts.  One of these days...
BTW, this particular project uses the RFM69 as an OOK transmitter and doesn't even use receive...

Good luck!
Tom

What timeout do you use if not fixed by TPL5010?

[TomWS]

What timeout do you use if not fixed by TPL5010?

I would say, in general, if your nodes' data isn't absolutely critical, then, for a gateway, I would use roughly 2.5 times the fastest reporting rate that you'll get from a node and then do a WDTreset() with every received packet.  If you're data is more critical AND you're not likely to miss a packet from a node (which, if it's critical, you shouldn't) then I'd set the time to slightly more than the time between packets and, again, do WDTreset on every received packet.  This way you shouldn't miss more than one packet (per node) if you get a hang.

I did make a breakout board (see below) that adds a TPL5010 to a Moteino by piggy backing over the lower set of pins.   It derives power from GND, and 3.3V pins, has direct connection to Reset pin, and spans up to D6 so that D3-D6 can be used for Wake & Done connection (depending on your other usage).  It also spans up to A7, but that's merely due to physical layout.  I can post the design if you want.

Tom

[LukaQ]

I would do SW reset, but this being such infrequent event, with no way of testing it (I don't think I would go so far as to bring out few 10kv inverter to make discharges neat it), to see if WDT in atmega is still working right (this is also considered hardware WDT right, just in IC instead of out?)
Data is not critical, weather data and some garage/lights... nothing important

I would be great to have maybe even RPi poll Moteino, if it is still there

[TomWS]

I would do SW reset, but this being such infrequent event, with no way of testing it

Yeah, how do you know if you're synthesizing the problem?

(I don't think I would go so far as to bring out few 10kv inverter to make discharges neat it),

LOL!  This reminds me of a project I was assigned to harden a piece of office electronics from static discharge. I was successful (won a corporate award for the effort), but I tic'd off every engineer in my lab as I ran my Van de Graaff generator during tests!  I also blew out the VP's expensive calculator as he challenged me to 'be need to be as good or better than this'.  The circuit was apparently better as it survived the blast

to see if WDT in atmega is still working right (this is also considered hardware WDT right, just in IC instead of out?)

I'm not sure I understand what you mean here...

I would be great to have maybe even RPi poll Moteino, if it is still there

That's a reasonable approach in this case, especially if you can touch the Reset pin if the moteino doesn't respond.  Also, I avoid WDT timer on Motes because the WDT timer gets used in LowPower delays so prone to ambiguous states.  If you're not using LowPower delays and can really dedicate the WDT  to Watchdog function, that should work too.

Tom

[LukaQ]

Yeah, how do you know if you're synthesizing the problem?

Can't say 100%, but 99.99% is pretty close right? Well, I wasn't doing anything at that time (both times), but both times it started to rain really bad with a lot of close by lightning. Apart from that, system was working with 100% uptime for several months, apart from my resets, to make changes go into effect.

I'm not sure I understand what you mean here...

There is no way 328's WDT could freeze, even if there is nothing going on with everything else(hard lockup)? I mean I could accept gateway Moteino being reset every 8s or multiple of that.

If you're not using LowPower delays and can really dedicate the WDT  to Watchdog function, that should work too.

I'm using standard gateway sketch provided by Felix, I'll say, I don't use it for delays on this mote

For polling, I think I could do something like that mote is doing, listening to serial and does something, like reset of dump flash (instead of flash dump, it could just send "i'm here")

[TomWS]

There is no way 328's WDT could freeze, even if there is nothing going on with everything else(hard lockup)? I mean I could accept gateway Moteino being reset every 8s or multiple of that.I'm using standard gateway sketch provided by Felix, I'll say, I don't use it for delays on this mote.

You are correct, if you're ONLY using the WDT as a Watchdog, then this should be reliable.  It's when it keeps getting switched between roles that problems happen.

For polling, I think I could do something like that mote is doing, listening to serial and does something, like reset of dump flash (instead of flash dump, it could just send "i'm here")

Then you'd need a daemon in the RPi to listen for the message and reset if it doesn't occur.  I'd be more inclined to use a cron task to ping the Moteino periodically and reset if it didn't get a response.  I'm not a big fan of daemons because you're introducing yet another piece of code that needs to keep running (and start reliably when the RPi reboots) and cron HAS to run!

Tom

[LukaQ]

Yes, I could use cron, but as of now, I have void as far as what it would do. Probably send serial and expect to get something back. But if it wouldn't, then what? toggle one of the pins, which would be connected to reset of mote?

[syrinxtech]

Can't say I've ever seen a problem with lightning per se, but then again, my base station controlling everything in the house is on a APC UPS.

[LukaQ]

I don't think that matters, it's not like power is dirty and it kinda is the same as if it went through online ups, since it has switching AC/DC before it. If power was a problem, PI would lock up or reset before moteino (which gets its power from the same 5v and I don't think RPi has more stored energy compared to consumption of it)

[LukaQ]

Tom,

do you have any of this boards laying around?

[TomWS]

Tom,

do you have any of this boards laying around?

If you mean the TPL Watchdog, yes, I have a few blanks.  PM me.

Tom