Wireless Boot in 2.5 Seconds / High-Speed Datastreams

[joelucid]

I take it that the only thing that need be incorporated into the main sketch--or the main library used by the sketch--is a way for you to wirelessly force a reboot?

Yeah exactly - that's one of the key benefits of this approach. You can install apps that know nothing about the deployment mechanism. Just assign the image to the node and plug it in.

I usually don't do anything automatically but manually decide which nodes I want to restart. For rapid development I just compile directly to the location where the gateway picks things up and send a reboot command to the node in question - all with one keystroke from emacs. I cross-compile from the mac so compilation is immediate. And then less then 4s later it's up on the node.

It's actually a fun way to see whether code changes caused binary changes: just compile and see if the node picks up a new version. Nice way to brush up on your C operator precedence skills ;-)

[joelucid]

and send a reboot command to the node in question

Of course it helps that the nodes respond immediately thanks to listen mode - you see how it all fits together :-)

[WhiteHare]

I don't recall you ever explaining why you preferred 3 seconds rather than the 3 minutes you favored before, but I suppose that explains it: if you're wanting to update a node, 3 seconds is a tolerable incremental delay, whereas at 3 minutes it would be so irksome you might opt instead to walk over and wire-up the connection (well, back when you still had headers to interface with, that is!).  Is that how you arrived at 3 seconds?  i.e. it's the tradeoff that best satisfies both your patience and your battery life requirements? 

[joelucid]

Is that how you arrived at 3 seconds?

I added listen mode to be able to interactively work with my nodes. I'll be on the command line and set parameters, send commands, read settings, update firmware etc. 3s was about the boundary where it still felt "live".

[Felix]

Hey Joe,
It's been a while, any chance you could release the wireless bootloader code?

[TD22057]

Hey Joe,
It's been a while, any chance you could release the wireless bootloader code?

+1 - I'd love to see this as well.

[executivul]

+1

[WhiteHare]

@joelucid

Given that your bootloader installation sketch presumably runs like an ordinary sketch, how is it able to write the new bootloader to the protected region of flash where the bootloader resides?

[joelucid]

how is it able to write the new bootloader to the protected region of flash where the bootloader resides

It just uses the old bootloader and itself to write the protected regions. Takes a couple of tricks to make it work - I think I've explained earlier how I do it.

Hey Joe,
It's been a while, any chance you could release the wireless bootloader code?

As I've explained earlier I would only consider making the bootloader available as part of a solution that just works. In particular it needs a backend.

For example I could create a device that attaches to your wifi, can be fed with new versions of firmware which it will then cause clients to install. That way the bootloader backend would be independent of whatever other gateway one uses. Or my backend could be integrated into Felix gateway, I suppose. That would be a narrower solution but one that could boost adoption of the gateway.

But both options are significant work which competes with my other projects - and since the bootloader already works for me they never quite seem to make it to the front of my priority list. Although I have come at least as far as ordering a esp8266 dev board which is now sitting here to be tried out as install gateway. So I guess there's hope. Just keep pestering me 

[executivul]

So I guess there's hope. Just keep pestering me 

My bootloader is almost working using a modified WirelessHEX69 lib integrated in optiboot and Felix's python script. It's about 6KB and still unreliable, not to mention I do not use any of your speed improvements yet.
Yours seems much better so I would be more than happy to use it instead.
Please release it in however form you consider, I must modify it anyway to match my atmega2560 nodes so it won't be quite plug&play.

Vielen Dank!

[joelucid]

It's about 6KB and still unreliable, not to mention I do not use any of your speed improvements yet.

Just a little progress report: I decided if I make this available I should optimize and clean it a little. Turns out there was still a LOT of dead wood around. Without coin cell support I'm at 2570 bytes now. With coin cell support it's 2912 bytes.

I've added what I think will become a pretty cool feature: every client gets assigned a worldwide unique id on first boot. Then the bootserver just assigns dynamic network node ids as clients boot up - sort of analogous to DHCP.

This will enable global node configuration later on - where I just configure my nodes once and they will feel at home with whatever gateway I connect them to. Be it one of several operating in a large area. Or just the case of my weekend home gw vs my home gw.

I've also started to add support for non 328p AVR's and have a Moteino Mega ordered from Felix so I can test that later on.

[executivul]

Wow. Impressive work.
Do you still have the serial support?

Now I'm struggling with ICSP while having the RFM connected. I use 5V Arduinos so a level converter is needed, but the OE for the level converter is active high on the 3.3V side, so while I have a pulldown for the RFM not to interfere with the ICSP it seems I also must use another pin for OE of the level converter through a voltage divider... lots of extra wires and components I'm not sure I want, maybe I can get away with some 1K-2K resistors in series with the SPI lines. Should buy some more spare time for testing but can't find it available anywhere 

[joelucid]

Do you still have the serial support?

No - I'm trying to make it as small as possible.

Now I'm struggling with ICSP while having the RFM connected. I use 5V Arduinos so a level converter is needed, but the god damn OE for the level converter is active high on the 3.3V side, so while I have a pulldown for the RFM not to interfere with the ICSP it seems I also must use another pin for OE of the level converter through a voltage divider... lots of extra wires and components I'm not sure I want, maybe I can get away with some 1K-2K resistors in series with the SPI lines. Should buy some more spare time for testing but can't find it available anywhere 

Gosh that sounds complicated. Maybe convert the Arduino to 3.3V? I see no advantage in running a mixed setup. https://learn.adafruit.com/arduino-tips-tricks-and-techniques/3-3v-conversion

[executivul]

Thanks for the link. Interesting read, but unfortunately mega 2560 requires 4.5V for stable 16Mhz operation. I guess 3.3V it's just too low to be considered safe. Using 3.6-4V just to force the RFM a little over max and the mega a little under min also seems like a recipe for bad things.

I need the 16Mhz since there are quite a few things to take care of:
-RFM69
-OLED
-6-8 Rotary encoders
-6-8 Relays
-2x PCA9685
-DS18B20 (local in wall temperature for safety)
-SI7021
-DHT22 (already have bought them, then decided to go Si7021 for better Hum accuracy but still keeping them for comparison sake)
-Analog light sensor
-Analog sound sensor

I have a mean loop time of 15-17ms with spikes as long as 300ms when reading some of the sensors. I guess I have to optimize some of the libs used or change them to state machines, use more interrupts and/or prioritize things.

[WhiteHare]

@executivul: Eventually loose wire connections are going to drive you mad, especially on some components (like the RFM69) that are so sensitive to slight voltage changes and noise. I do wish the diameter of the dupont male wire connectors was just slightly larger (or the female slightly smaller) so that the connections would be more snug. FWIW, I switched to soldering as much as I can as soon as I can, and it has been a lot less frustrating (though obviously more time consuming).