Author Topic: Wireless Boot in 2.5 Seconds / High-Speed Datastreams  (Read 112497 times)

joelucid

  • Hero Member
  • *****
  • Posts: 868
Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
« Reply #30 on: January 13, 2016, 04:33:19 AM »
Hi Mircea,

sorry for the late reply.

I've periodically been thinking about how to release the bootloader. Doing so would obviously require work and so I only want to do it if it is really useful to many people. Currently it's heavily tied into my infrastructure and I don't think it's useful to just dump all my code on people (I did that to Tom and even he lost interest after the drop ;D).

I think it first requires some install server that people could use (either you just plug that into USB and program like an Arduino and it relays the program to the Moteino to be programmed, or it's a standalone server along the lines of the minimal gateway (esp8266 + rfm69w) we just discussed in another thread). Obviously the backend code could also just be integrated into Felix gateway but I find that pretty restrictive.

Then there would need to be a way for Moteino users to install it (bootstrap bootloader).

So overall: a lot of work. I currently have too many interesting other projects going on to make it a priority.

Joe

TomWS

  • Hero Member
  • *****
  • Posts: 1930
Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
« Reply #31 on: January 13, 2016, 08:12:05 AM »
(I did that to Tom and even he lost interest after the drop ;D).
Not true!  As you cite below, I have too many other interesting things going on to tackle this problem.  Especially when my code never needs to be updated!   :o
uh, JK!

Quote
So overall: a lot of work. I currently have too many interesting other projects going on to make it a priority.
Tom

PS: I predict that 'something' will happen 'sometime' this year on this matter.  It's far too interesting to disappear.

executivul

  • NewMember
  • *
  • Posts: 48
  • Country: ro
Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
« Reply #32 on: January 13, 2016, 09:42:45 AM »
Thank you Joe.
I'm a hardware engineer with some programming skills. I mean I can understand a piece of code unless it uses pointers to pointers referencing other pointers and tend to do things using simple for/while/if statements instead of #define code art.
Now I see that I can reserve 4K for the BL using fuses, which for me on 2560 is next to nothing. I would prefer reserving 16K for a bootloader instead of trying to optimize things, but 4K is the limit it seems.
Also all my nodes run wall powered for now, so power saving as long as it doesn't glow red hot it's nothing to bother for now.
I was contemplating having more networks, but it seems having a single one is simpler, still waiting for some parts from the far east to build more nodes and test rf crowding and rf separation for multiple networks running in the same space.
For now the plan is: the sketch sets the network parameters in EEPROM and the BL reads from there, tries to find a server/update, if available download and flash then boot. 2-3-5-10 second boot delay is not a problem either.

If it's possible just drop the code on me :) of course after removing all sensitive security data.
If I can use it I'll let you all guys know, though it might get a little less readable after I do my things. For example I took the clickEncoder library and because it didn't work with multiple encoders I stripped all the classes, functions, unnecessary variables and fancy coding, it's reduced now to just two functions and works for me, but for somebody having input active high it's harder to find and change the code than it was in it's original form. I'm a little code barbarian so to speak.

Best wishes.

executivul

  • NewMember
  • *
  • Posts: 48
  • Country: ro
Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
« Reply #33 on: February 21, 2016, 05:07:06 PM »
I'm still around :)
I've managed to compile an optiboot for my mega 2560 and I'm able to program via USB/UART up to 128KWords to it and it's all ok. I compile optiboot on OSX Capitan, avr-gcc from Arduino 1.6.7, make tools from Xcode (CrossPack failed me). Upload the BL via avrdude+AVRDragon. 622bytes  8)

Now the hard part: porting the RFM69.

Joe how did you manage to port the RFM69 library to the optiboot build environment? Did you strip all the classes and all the Arduino helpers (Serial.write, digitalWrite, etc) and re-made it in simple C or did you chose avr-g++ for optiboot and included everything from the Arduino environment? What size is your final bootloader?

Thanks.

joelucid

  • Hero Member
  • *****
  • Posts: 868
Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
« Reply #34 on: February 23, 2016, 02:30:13 AM »
Yeah I stripped all those arduino helpers. Delay is also gone. Also I made all methods static with global instead of instance vars to avoid having to pass a this pointer around. It's still C++, but only in name since all methods are static, no virtuals etc. I'm polling instead of interrupts. And I optimized all around making it as small as possible.

My bootloader is exactly 4k. Every new piece I add I have to hunt somewhere to cut.

executivul

  • NewMember
  • *
  • Posts: 48
  • Country: ro
Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
« Reply #35 on: February 23, 2016, 05:35:22 AM »
By "Delay is also gone." you mean by any chance millis()? That's what I'm struggling with for now.
Polling instead of moving interrupts vectors to BL is the next on the list. The SPI interrupt seems the most troublesome.

Size is still a problem: on Mega 2560 I can have 8kB BL, but I also intend to use a bunch of 328p with just 4kB.
The simplest struct_send.ino compiles into 10kB if I use all the Arduino bells and whistles so that's a no no.

Thank you! Appreciate it.

joelucid

  • Hero Member
  • *****
  • Posts: 868
Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
« Reply #36 on: February 23, 2016, 07:00:27 AM »
Yeah I meant millis. Since we only need to measure short periods no overflow is needed and you can easily just use one of the native timers instead.

executivul

  • NewMember
  • *
  • Posts: 48
  • Country: ro
Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
« Reply #37 on: February 24, 2016, 07:01:52 AM »
Look what I found: https://github.com/UKHASnet/ukhasnet-rfm69 (My rant about reinventing the wheel.)
It's a RFM69 + SPI library based on simple AVR calls, no Arduino "boiler plate" :)
Unfortunately I'm away and I can not compile it for the next few days...

Excuse me Felix if it's not appropriate to post this here.

Felix

  • Administrator
  • Hero Member
  • *****
  • Posts: 6866
  • Country: us
    • LowPowerLab
Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
« Reply #38 on: February 24, 2016, 09:09:55 AM »
No problem, I am aware of UKHASnet, niec project, they have some nice info.

executivul

  • NewMember
  • *
  • Posts: 48
  • Country: ro
Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
« Reply #39 on: March 02, 2016, 09:10:32 AM »
By the way Joe, I liked so much your idea of CRCing the Flash and checking against server CRC that I decided to use it and get rid of all the versioning :)
But how do you know where to stop CRCing on the AVR?
I mean I can compute CRC from 0 to 0x1EFFF (in my case) that's all the flash except the BL, but I must then adjust the HEX CRC accordingly since the HEX does not contain all available addresses.
Or I can start from 0 and stop at 0xFF since unwritten flash is 0xFF but how long a string of 0xFF should be to be safely considered "past the end in no man's land"?
Or it's better that server sends the HEX's length in bytes and locally we CRC from 0 to the length received, assuming the new sketch is not identical and only longer?

Thanks.

joelucid

  • Hero Member
  • *****
  • Posts: 868
Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
« Reply #40 on: March 02, 2016, 09:59:39 AM »
Yeah I think the CRC approach is really essential to making the bootloader resilient. Whatever happens - whether there's a transfer error, whether I change the hex file during update, whether there's a bootloader bug - in the end the app is started exactly if it got installed correctly. Otherwise I just try again.

I create two CRC's: one from 0x0000-0x6fff (the app) and one from 0x7000-0x7fff (the bootloader). On the server I just load the hex files into a similar sized region that I've previously filled with 0xff. Which leaves everything at 0xff unless it's present in the hex file. However I only transmit pages actually within the hex file if an update is necessary to minimize transfer time.

On the client you just need to erase all flash pages that aren't in the update prior to an update so that the CRC's match up again after install. Both sides just CRC the entire region then.

The reason I prefer to CRC the full regions is that I can transmit the CRC's with every bootloader request, including the first. No need to ask which pages to sum over before transmitting the CRC's. And having the CRC's in every call to the server allows the server to be completely stateless, serving multiple clients concurrently. I can restart the server during an update and everything proceeds just fine.

Joe

joelucid

  • Hero Member
  • *****
  • Posts: 868
Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
« Reply #41 on: March 02, 2016, 10:14:00 AM »
Quote
allows the server to be completely stateless

BTW, not only the server. The clients are also almost completely stateless. Partial install? Just restart, calculate new CRC and try again.

There's one bit of state which I do keep: whether the server has at one time approved what's currently installed on the client as not needing an update. I reset that flag when the server tells me I need to update. After installation is complete I just start another installation. If the previous install was successful the server will return that no update is needed and I set the flag before starting the app.

That flag allows one very convenient feature: a client can start its app if it's previously been approved by the server even if it can't reach the server during startup. That's possible because the flag allows a client to differentiate between an app that might merely be out of date and one that might be corrupted in absence of server communication.

WhiteHare

  • Hero Member
  • *****
  • Posts: 1300
  • Country: us
Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
« Reply #42 on: March 02, 2016, 11:29:25 AM »
If you're OTA updating the bootloader itself, is the new bootloader code saved in the space that's normally reserved for the sketch until the new bootloader code is validated and then used to overwrite the bootloader in its special part of flash memory?  That's clever.  Then on reboot the bootloader just calls for the sketch, as it apparently always does, and it just keeps calling until it gets a valid sketch. So, that's how you are guaranteed to avoid bricking a node.   Nice! 
« Last Edit: March 02, 2016, 11:42:05 AM by WhiteHare »

joelucid

  • Hero Member
  • *****
  • Posts: 868
Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
« Reply #43 on: March 02, 2016, 11:49:10 AM »
Quote
If you're OTA updating the bootloader itself, is the new bootloader code saved in the space that's normally reserved for the sketch until the new bootloader code is validated and then used to overwrite the bootloader in its special part of flash memory?  That's clever.  Then on reboot the bootloader just calls for the sketch, as it apparently always does, and it just keeps calling until it gets it.

Exactly. Or more simply stated: a bootloader is replaced by an app that contains and installs the new bootloader (the "bootloader installer"). When the server sees a bootloader CRC which doesn't match it will consider the right app for the client to be the bootloader installer. Therefore the app CRC's don't match anymore leading to an app install of the bootloader installer. Once the bootloader installer is downloaded a restart happens and if the client's app CRC matches the bootloader installer CRC it gets executed and replaces the bootloader.

Another restart happens and the bootloader CRC's now match again so the right app for the client is it's previous app. Therefore the app CRC's don't match and the old app gets reinstalled.

Really pretty simple but it covers all cases and there is no chance for corruption or bricking other than installing a buggy bootloader. I'm running this stuff day in day out and it's so robust that I don't even put ISP headers on my boards.

WhiteHare

  • Hero Member
  • *****
  • Posts: 1300
  • Country: us
Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
« Reply #44 on: March 02, 2016, 12:43:33 PM »
Very elegant.  I really do hope this eventually becomes a standard way of doing things. 

To facilitate updating a sketch, I take it that the only thing that need be incorporated into the main sketch--or the main library used by the sketch--is a way for you to wirelessly force a reboot?  Or, instead, do you just have the node reboot every now and then and pick up the latest sketch that way?  Or I suppose it could be both ways, which would satisfy both the desire for immediate updates on a particular node (e.g. during a  rapid development cycle) as well as distributing less critical updates to all nodes.

« Last Edit: March 02, 2016, 12:49:46 PM by WhiteHare »