LowPowerLab Forum

Software support => Wireless Programming (aka OTA) => Topic started by: joelucid on March 19, 2015, 12:31:16 PM

Title: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 19, 2015, 12:31:16 PM
Hi guys,

I've been using a bunch of Moteinos lately - great platform! With the first 10 in use as various sensors / controllers throughout the house I got annoyed by having to run around to reprogram them. I also found the programming time of  >10s a major turn-off.

So I thought I'd give Felix wireless programming a try and ordered the next 5 Moteinos with flash. Thankfully I live in Germany and any order takes about 14 days to arrive here - so I started to experiment with alternatives.

In the end I put together a wireless bootloader that installs from an Raspberry Pi based install server directly to the atmega328p flash - no extra chip required. I also provide a feature in the bootloader that allows an app to install an updated bootloader - so you really never have to physically get to your nodes again.

Since I wanted it quick I did the following:

- Asynchronous protocol

Latency quickly brings down performance with handshake-heavy protocols. So I'm just streaming over as many packets as fit into the 328p RAM and then wait for one response saying what arrived. That brings you down to ~14 acks for a 20k payload - much better than handshaking each packet

- Higher bandwidth settings

I found a set of RFM69 settings that allow fairly reliable transmission at 200kbit throughout the house. Here are the important parameters:

    /* 0x03 */ { REG_BITRATEMSB, RF_BITRATEMSB_200000},
    /* 0x04 */ { REG_BITRATELSB, RF_BITRATELSB_200000},
    /* 0x05 */ { REG_FDEVMSB, RF_FDEVMSB_300000},
    /* 0x06 */ { REG_FDEVLSB, RF_FDEVLSB_300000},
    /* 0x19 */ { REG_RXBW, RF_RXBW_DCCFREQ_000 | RF_RXBW_MANT_20 | RF_RXBW_EXP_0 },

- Transmit / Receive without Standby Pauses

If you send multiple packets in a row it is much faster to keep the radio in transmit mode and send one packet after the other than to switch off the transmitter between packets. I rewrote the RFM69 library to support these streaming cases. I also eliminated the interrupt handler since it kept causing me problems. I now poll instead.

When you don't go into Standby on the receiver side you also don't need to set RF_PACKET2_AUTORXRESTART which saves additional time each packet.

- RF_PACKET1_DCFREE_WHITENING

This parameter needs to be switched on to enable whitening of the data packets by the radio. Otherwise empty pages (0xff) will cause lots of dropped packets - it seems the radio needs some change in the data stream to work effectively.

- No busy wait in sendFrame

I've eliminated all the busy waits. No canSend, no busy wait for transmission ready.

----

With these changes (and I'm sure some I'm forgetting) I can transfer 20k of code in about 1.5 seconds. Together with the page erasing and flashing in the bootloader I flash 20k of code wirelessly in 2.5 seconds.

As for size this is now around 3.5k for the bootloader. I'm pretty sure I could get it under 2k with effort but I don't need that yet. I also plan to add serial boot capabilities to the bootloader using the same protocol so I can boot the wired gateway Moteinos from the install server, so will probably end as 4k bootloader anyway.

The bootloader transmits a node ID, CRC of itself and of the application on each boot and the bootserver sends updated versions as needed. Apps are sent directly to the bootloader for flashing. If the bootloader needs updating an app that flashes the new bootloader gets sent. Once that's done the app gets installed again.

For robustness the bootloader / bootserver are completely seperate from RFM69. But I'm using standard RFM69 for apps. Therefore protocol changes can be implemented at the app level without impacting the boot system (in fact I reenabled encryption at the app level again yesterday without a glitch for booting). The boot server uses the same Moteino as the gateway and shares the radio. The bootserver receive gets 15 ms every 500 ms - or a 3% duty cycle of the radio, without impact for sensor data transmissions.

All in all this works reasonably well. I hope this will be helpful to others implementing streaming applications or interested in wireless bootloaders. Let me know what you think.

I'm sure the speed could be tuned quite a bit more - eg by using 255 byte data packets rather than the 64 bytes required by encryption. However I'm now happy with what I have and will focus in the immediate future on putting the next 12 Moteinos which have finally arrived to use :)

Joe
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 19, 2015, 12:31:50 PM
BTW one change request for Felix:

If I reinitialize the RFM69 lib after running my radio code in the bootloader sometimes an interrupt gets triggered in the RFM69 constructor. At that point selfPointer isn't yet set so the application restarts. What helps is to set

RFM69* RFM69::selfPointer = 0;

and then:

void RFM69::isr0() {
    if( selfPointer ) selfPointer->interruptHandler();
}

This probably best since we don't want a virtual function to be called in the ctor anyway. Could you add that so I don't lose it with the next library update?
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: TomWS on March 19, 2015, 04:42:02 PM
Hi guys,

I've been using a bunch of Moteinos lately - great platform! With the first 10 in use as various sensors / controllers throughout the house I got annoyed by having to run around to reprogram them. I also found the programming time of  >10s a major turn-off.

So I thought I'd give Felix wireless programming a try and ordered the next 5 Moteinos with flash. Thankfully I live in Germany and any order takes about 14 days to arrive here - so I started to experiment with alternatives.

In the end I put together a wireless bootloader that installs from an Raspberry Pi based install server directly to the atmega328p flash - no extra chip required. I also provide a feature in the bootloader that allows an app to install an updated bootloader - so you really never have to physically get to your nodes again.

<snip...>
Ok, let's see if I understand what you've done.   Paraphrasing what I 'heard':

You've modified Moteino nodes with a new bootloader.

On reset, the new bootloader does not run from flash, but, instead, contacts your gateway at the RaspPi and downloads an image?  What initiates this decision?

The new bootloader streams its image from the server, stuffs it into flash (presumably at its final address?), and then boots to it once the image is completely downloaded?

I'm sure I've missed something, but that's the general idea?

Tom
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 19, 2015, 05:15:40 PM
Yeah that's basically it.

Obviously the bootloader does run from flash. But it then contacts the server on every boot to potentially update the application or itself. Whether a new application or bootloader image is required is determined by matching the CRC16's of the images currently installed on the client with the ones set up on the server for the booting node. After application install or timeout the app is run.

Since the bootloader installs the app in the application section it's not required to install it via flash chip to avoid bricking the moteino: if the install fails the checksums will not match on restart and the installation will just be repeated. If not the app but the bootloader is updated that happens by installing an app which will then burn the bootloader. Since the app install is safe the bootloader install also requires no external flash (if the bootloader installation app install fails it will just be repeated).
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: TomWS on March 19, 2015, 05:50:31 PM
Yeah that's basically it.

Obviously the bootloader does run from flash. But it then contacts the server on every boot to potentially update the application or itself. Whether a new application or bootloader image is required is determined by matching the CRC16's of the images currently installed on the client with the ones set up on the server for the booting node. After application install or timeout the app is run.

Since the bootloader installs the app in the application section it's not required to install it via flash chip to avoid bricking the moteino: if the install fails the checksums will not match on restart and the installation will just be repeated. If not the app but the bootloader is updated that happens by installing an app which will then burn the bootloader. Since the app install is safe the bootloader install also requires no external flash (if the bootloader installation app install fails it will just be repeated).
I don't want to pick on your last statement, because I think it's a second order problem (we wouldn't HAVE to update the bootloader this way), however, I'm trying to understand why you think loading a new bootloader is failsafe.  If the bootloader is being overwritten during download, and doesn't succeed, the app could try to reload it, but, if, for some reason, the system resets before the successful download then isn't the node bricked?  Like I said, this is not something I would lose sleep over, but if I'm missing something I'd like to know.

Re your streaming method, I like this idea for powered nodes. I'm struggling with whether this is energy efficient for battery operated nodes.  I suppose if we can determine that the receiving node is rarely transmitting, then this certainly offsets the sustained awake time on receive.

Where is your code?

Tom
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 20, 2015, 02:46:41 AM
Tom,

rg the bootloader update: I've written a minimal app that flashes a bootloader. If the server finds that a new bootloader has to be installed it takes that app and merges it with the new bootloader (essentially it just takes the bootloader code which resides at 0x7000-0x7fff and appends it to the app at 0x6000). The resulting app can install the bootloader without any wireless access since it already contains the bootloader code. Therefore the normal failsafe app install process guarantees failsafe transmission of the bootloader.

Rg battery: to minimize battery use you want to minimize send/receive/wake time. No better way to do that than flashing the whole app in 2.5 seconds. And yes, the receiving node transmits very rarely since it doesn't ack each package (only 1 ack every 24 packets at the moment).

The code isn't public yet. If there's enough interest I might make it available.

Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: kobuki on March 20, 2015, 06:04:36 AM
I'm certainly interested. Most sketches are perfectly fine on the available 28k. The ability to program a node remotely without a flash part that's only used for remote flashing twice a year is very nice. I think the transmit rate could be lower to make transmission more stable. I know you use a CRC and it works perfectly within your house but such a high rate limits the programming distance of the nodes, while the most important usage scenario is reprogramming nodes at remote, inconvenient places, out in the field, etc. where they are not as close.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: Felix on March 20, 2015, 08:39:59 AM
Joe, this all sounds fantastic. However I've read and re-read your post and I may be missing the obvious however I wonder how you'd deal with the fact that you need your packets to arrive in perfect order. Any dropped packets will corrupt the entire image. In my scheme, the external memory is used as temporary storage to ensure the entire image is received as expected, before moving it to its final destination in the AVR flash memory.
And if I understand this right, it's the node that initiates the download (ie gateway must have the image ready when the node wants it), which is exactly the reverse of what my scheme has - the node must be ready when a programmer node (typically not the gateway) wants to reflash it. This may not work in all situations. In my Moteino Framework, one idea was decentralization and autonomy, hence the wireless programming scheme I adopted. A small bootloader that is lean and doesn't depend on a certain radio implementation is also essential for this requirement, again a difference in purpose.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: TomWS on March 20, 2015, 08:40:55 AM
Tom,

rg the bootloader update: I've written a minimal app that flashes a bootloader. If the server finds that a new bootloader has to be installed it takes that app and merges it with the new bootloader (essentially it just takes the bootloader code which resides at 0x7000-0x7fff and appends it to the app at 0x6000). The resulting app can install the bootloader without any wireless access since it already contains the bootloader code. Therefore the normal failsafe app install process guarantees failsafe transmission of the bootloader.
If I understand correctly, you're saying the new bootloader is appended to an application which, if successfully loaded, will run and copy the bootloader to its final address?  If so, yes, that should work nicely.

Rg battery: to minimize battery use you want to minimize send/receive/wake time. No better way to do that than flashing the whole app in 2.5 seconds. And yes, the receiving node transmits very rarely since it doesn't ack each package (only 1 ack every 24 packets at the moment).
The problem is the internal resistance of some batteries, eg CR2032.  It will work fine with 130mA peaks for 4mS if the proper capacitor is used.  It will not work at all if drained at an average of 16mA for 2.5 seconds.

The code isn't public yet. If there's enough interest I might make it available.
Fascinating, let us know when it's available.

Tom
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 20, 2015, 09:13:40 AM
@Felix, the protocol works like this: the bootserver sends 24 packets to the client and then waits for an ack which contains exactly which packets where received at the client. The client receives until it doesn't get a packet for 10 ms and then sends the ack telling the server what he got. The server resends any packets that weren't received by the client in the next packet burst.

As you see it's completely fault tolerant. With my own rfm69 code I typically lose less than 1% of the packets if I only transfer. That increases to between 1% and 2% if I program the flash because it takes variable time to program and so I sometimes drop packets.

Rg. the reversed roles of client/server: the server doesn't need to have the image ready at client boot time - in that case the client times out and starts its old version. I do support the server pushing an update to the client at the app level - basically I set up the new image and tell the client to reboot. In general my battery clients can't afford to keep the receiver on so they check in periodically with the server to report data or receive commands. Of course wired clients can keep the receiver on and allow for instantaneous push updates.

Having a simple and tiny bootloader is clearly a strong advantage of your solution. You buy that by putting the wireless programming support on the app level which has a couple of drawbacks: (a) A broken app will brick the moteino, (b) can't update the bootloader wirelessly, (c) standard Arduino Hex files can't be installed since they don't include the wireless programming support (d) flash chip required.

Joe
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 20, 2015, 09:23:14 AM
@Tom, I think you've got it.

Interesting point about the internal resistance of some batteries. I haven't had that issue yet. Clearly if you can't take the whole app in one stream a bootloader based solution is no good and the flash chip / trickle download approach is preferable.

@Kobuki, yes, it would be nice to support slower bitrates, too, for more remote clients. A big motivator for me to work on this was to get programming turnaround times as short as possible - so I've tried to get the transfer speed as high as I could. It would be nice if the protocol could gracefully degrade down to lower speeds if RSSI or drop rates indicate probems.

BTW I noticed that drop rates drastically came down as I changed my code to stay in transmit for each packet burst. I haven't tested this in more difficult scenarios yet, but I suspect results could be better than using stock RFM69 for streaming data.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: Felix on March 20, 2015, 11:04:34 AM
@Felix, the protocol works like this: the bootserver sends 24 packets to the client and then waits for an ack which contains exactly which packets where received at the client. The client receives until it doesn't get a packet for 10 ms and then sends the ack telling the server what he got. The server resends any packets that weren't received by the client in the next packet burst.

As you see it's completely fault tolerant.

So you write the packets as you get them into the flash because there is no temporary storage. The moment you write a single packet, the assumption is that you will get the whole image, there is no going back. If something happens in between and transmission drops the target is bricked. But the assumption is this will not happen in the short few seconds of transmission.

Having a simple and tiny bootloader is clearly a strong advantage of your solution. You buy that by putting the wireless programming support on the app level which has a couple of drawbacks: (a) A broken app will brick the moteino, (b) can't update the bootloader wirelessly, (c) standard Arduino Hex files can't be installed since they don't include the wireless programming support (d) flash chip required.
A potential problem with a custom bootloader is that there is duplication of code for the radio, increases the size multi fold, the masses will complain, more so if the size is considerably larger than the stock optiboot (512b).
I believe the bootloader should be write-protected at all times, this ensures the bootloader section cannot be overwritten by accident. This has also happened in the past when fuses were not set and the result was a bricked unit.

My opinion is that your solution will work great for those who can implement it in their project constraints and need a tight coupling between the application and bootloader, and where the special character of the bootloader will fit their requirements well.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: kobuki on March 20, 2015, 11:19:35 AM
@Felix: I'd like to note that the footprint of your OTA flashing offering is quite significant, and it adds to the radio code which is always needed. I think that anyone who can afford using that, can afford the 4k booloader - and this alternative solution doesn't need an additional flash chip. OTOH, if I understood correctly, joelucid ensures that all blocks are written safely before rebooting the Atmega, using that TCP-like ACK algo and by employing CRC16 checking. Of course, there are many ways his solution can be refined. I'd like to see his code so we can dissect it and make suggestions for the improvements already ;)
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 20, 2015, 02:29:21 PM
@Felix, all pages are verified via CRC's before they are written to flash. So corruption can only occur if the transfer gets interrupted - say because of power failure. if that happens it's no problem: on next boot the app checksum is calculated and will not match the server checksum since the flash process was interrupted. So the bootloader just starts another download.

As for bootloader protection: I have the fuses set similar to what you have. What I do is I provide a method eraseAndProgramPage in the bootloader via an API that's accessible for applications. The app can use that to flash the bootloader - not something that will happen by accident. [Actually it's not quite that simple since you need a way to flash the bootloader page that contains the eraseAndProgramPage code. I've created two implementations of the method on different pages in the bootloader. The app burns all bootloader pages but the one that contains one of the methods. Then as last step it uses one of the two implementation of eraseAndProgramPage in the newly flashed bootloader to burn the last page.]

I've thought about the size issue a lot. It's possible to call bootloader code from an app so I had planned to do that to save space for the RFM69 code - but then as my implementation diverged that became unfeasible. On the other hand I was able to reduce the size of the radio code quite a bit by eliminating what I didn't need, throwing out the Arduino boiler plate code, making all methods static etc. There's really not that much left there.

Note that the bootloader and the app are not tightly coupled at all. They are so independent of each other that you can install regular sample Arduino apps that don't know anything about the bootloader. I'd argue that's less tightly coupled than your solution.

I need to find time to publish the code so you guys can check it out.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: TomWS on March 20, 2015, 03:27:32 PM
<...snip>
A potential problem with a custom bootloader is that there is duplication of code for the radio, increases the size multi fold, the masses will complain, more so if the size is considerably larger than the stock optiboot (512b).
<snip...>
We should take a poll on how much program storage our apps take.  In my experience, (and I've got some FAT apps  ;) , I don't come anywhere near filling up 32K of program space.  RAM, yes!  I'd really like to see a new version of the 328P with double or more of RAM, but program space, I'm not sure it's an issue.  And, as Joe points out, his slimmed down RFM69 code is probably smaller than the Wireless Programming library.

Does anyone know how to set up a poll?

I'd like to gather (for 328 type procs only - don't care about Mega, they have PLENTY!):
Program Storage: max, min, average
RAM (not including heap): Max, Min, average

Tom
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: kobuki on March 20, 2015, 03:58:25 PM
Instead of starting a new thread, start a new poll, using the button at the bottom. I think you can only create simple multiple choices polls. The most important is the largest possible size so something like Up to 5, 10, 15, 25, and "I need all of the RAM dammit" or something similar should be good.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: TomWS on March 21, 2015, 08:30:12 AM
<...snip>

Note that the bootloader and the app are not tightly coupled at all. They are so independent of each other that you can install regular sample Arduino apps that don't know anything about the bootloader. I'd argue that's less tightly coupled than your solution.
I was thinking about this aspect and see that you did mention it.  So, from a mote's perspective, assuming that you're willing to pull it out of the wall one time and reprogram the bootloader (something not many people have done before), there would be NO change to existing apps other than removing the Wireless Programming library while keeping a snippet of code to reboot on demand.

The real burden would be on the gateway which would have to support two different wireless protocols (assuming you didn't have a dedicated reprogramming Gateway).  Have you looked at a gateway implementation that supports both (current RFM69 library and the code needed to reprogram on boot)?  I can't help thinking these would be mutually exclusive...
I need to find time to publish the code so you guys can check it out.
Yeah, that's a good idea...

Tom
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 21, 2015, 06:10:20 PM
@Tom, yes, correct on the ease of use of the bootloader approach.

My raspberry uses one Moteino for both boot and regular messaging. The way that works is that the Moteino listens for normal packets 97% of the time and for boot packets the remaining 3%. Since a booting Moteino will persistently ping the server for about a second such a low allocation to boot packet listening works. Apps don't notice it at all - apart from a tiny increase in retransmits.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: TomWS on March 21, 2015, 06:56:12 PM
@Tom, yes, correct on the ease of use of the bootloader approach.

My raspberry uses one Moteino for both boot and regular messaging. The way that works is that the Moteino listens for normal packets 97% of the time and for boot packets the remaining 3%. Since a booting Moteino will persistently ping the server for about a second such a low allocation to boot packet listening works. Apps don't notice it at all - apart from a tiny increase in retransmits.
Interesting.  What is the interval between switching modes, ie, how long is 97% and how long is 3%?   And you can switch modes of the radio that often without breaking anything?

Also, I was having some more thoughts on having to pull out a mote that's already been installed with the existing library.  Would you be able to load a new app using the current wireless programming library (since that's how the installed motes are configured) and that new app has the new bootloader attached and can load it? 

That load would obviously take longer than what you're talking about, but presumably you'd only have to do it once and you wouldn't have to uninstall an existing mote (a HUGE plus in my book!).

Tom
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 22, 2015, 01:36:01 AM
@Tom, I currently use 550 : 15 ms. I haven't had a problem with it other than the one with the Irq handler being called in initialize (see second post this thread).

Unfortunately you can't change the bootloader without physical access to the moteino unless specifically enabled by the previous bootloader (or with lax fuse settings which would expose you to the risk of accidentally bricking the device). That really is a pain since it sort of forces you to solder on headers just for the initial boot programming - even if you're just building an otherwise tiny thermometer.

At least with my bootloader you only have to do it once.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: KanyonKris on March 22, 2015, 01:47:31 AM
joelucid, excellent work! While your system may not work for very low power nodes (like those powered by coin cells, as TomWS pointed out), I can see your fast wireless reprogramming being very useful, especially when the system is under heavy development and needing many changes. I hope you will post your code so we can see the details of what you've done, and so others can use your method.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: ivars211 on March 23, 2015, 09:07:52 AM
Sounds really intriguing. Any chance of seeing the source code?
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 23, 2015, 02:14:49 PM
I'm planning on making it available - but I need to do some cleanup before.

I used the bootloader in my weekend house this weekend and had significant transfer issues. It's really strange: here in Berlin I live in an apartment that has such massive walls that I need two wifi basestations to penetrate everything. Yet even at Powerlevel 0 my RFM69hw installs at 300kbit without a glitch everywhere.

At the weekend home even at powerlevel 31 more distant nodes don't get a response from the bootserver even though the house is made of wood and wifi has no problem at all.

Anyway I'm implementing graceful bitrate reduction now. On the upside I found a 300kbit setting that's even faster than what I had. Will probably do 300kbit -> 200kbit -> 55555 -> 19200. When that works and I have the code clean I plan to make it public.

 
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: kobuki on August 30, 2015, 11:50:43 AM
Joelucid, have you ever released the source code for this downloader? Sorry if it escaped my attention before but I haven't been able to find it anywhere yet.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: WhiteHare on August 30, 2015, 01:46:32 PM
Joelucid, have you ever released the source code for this downloader? Sorry if it escaped my attention before but I haven't been able to find it anywhere yet.

+1
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on August 30, 2015, 07:02:00 PM
I've completely rewritten it for use with coin cell motes so no release yet. It's starting to work again, still fixing lots of bugs though. But it definitely works well with coin cells now.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: WhiteHare on November 22, 2015, 11:39:49 AM
@Felix,

Setting aside the bootloader issue and Joe's alternate paradigm, it would seem you could nonetheless adopt the majority of Joe's suggestions (below) into your existing wireless programming approach (i.e. using extra flash memory) so as to do much faster wireless programming, and you would incur no extra risks by doing so.  Even just by itself, the faster wireless programming speed would be a nice improvement.   Yes? :)

Since I wanted it quick I did the following:

- Asynchronous protocol

Latency quickly brings down performance with handshake-heavy protocols. So I'm just streaming over as many packets as fit into the 328p RAM and then wait for one response saying what arrived. That brings you down to ~14 acks for a 20k payload - much better than handshaking each packet

- Higher bandwidth settings

I found a set of RFM69 settings that allow fairly reliable transmission at 200kbit throughout the house. Here are the important parameters:

    /* 0x03 */ { REG_BITRATEMSB, RF_BITRATEMSB_200000},
    /* 0x04 */ { REG_BITRATELSB, RF_BITRATELSB_200000},
    /* 0x05 */ { REG_FDEVMSB, RF_FDEVMSB_300000},
    /* 0x06 */ { REG_FDEVLSB, RF_FDEVLSB_300000},
    /* 0x19 */ { REG_RXBW, RF_RXBW_DCCFREQ_000 | RF_RXBW_MANT_20 | RF_RXBW_EXP_0 },

- Transmit / Receive without Standby Pauses

If you send multiple packets in a row it is much faster to keep the radio in transmit mode and send one packet after the other than to switch off the transmitter between packets. I rewrote the RFM69 library to support these streaming cases. I also eliminated the interrupt handler since it kept causing me problems. I now poll instead.

When you don't go into Standby on the receiver side you also don't need to set RF_PACKET2_AUTORXRESTART which saves additional time each packet.

- RF_PACKET1_DCFREE_WHITENING

This parameter needs to be switched on to enable whitening of the data packets by the radio. Otherwise empty pages (0xff) will cause lots of dropped packets - it seems the radio needs some change in the data stream to work effectively.

- No busy wait in sendFrame

I've eliminated all the busy waits. No canSend, no busy wait for transmission ready.

----

With these changes (and I'm sure some I'm forgetting) I can transfer 20k of code in about 1.5 seconds. Together with the page erasing and flashing in the bootloader I flash 20k of code wirelessly in 2.5 seconds.

Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: Felix on November 22, 2015, 07:13:11 PM
@Felix,

Setting aside the bootloader issue and Joe's alternate paradigm, it would seem you could nonetheless adopt the majority of Joe's suggestions (below) into your existing wireless programming approach (i.e. using extra flash memory) so as to do much faster wireless programming, and you would incur no extra risks by doing so.  Even just by itself, the faster wireless programming speed would be a nice improvement.   Yes? :)

Yes, it's something to really consider, however if I put this on my to-do list I will get to it sometime in the near future.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: TomWS on November 22, 2015, 09:43:01 PM
@Felix,

Setting aside the bootloader issue and Joe's alternate paradigm, it would seem you could nonetheless adopt the majority of Joe's suggestions (below) into your existing wireless programming approach (i.e. using extra flash memory) so as to do much faster wireless programming, and you would incur no extra risks by doing so.  Even just by itself, the faster wireless programming speed would be a nice improvement.   Yes? :)

Yes, it's something to really consider, however if I put this on my to-do list I will get to it sometime in the near future.
Looks like a golden opportunity for some clever person to step in and provide a solution! 
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: executivul on January 12, 2016, 08:24:01 AM
Hello to all of you,
I'm building the automation for the house of my dreams. In this process I've come across this great site, thanks Felix.
Unfortunately I'm not using Moteinos but Mega 2560 for my project since I need far mor GPIOs than a 328p would offer, I need some 6/8 channel light controls with rotary encoders on each of them, 8 relay board, DHT, light sensor, sound sensor, PIR, and 8+1 RGB LEDs (8 for the rotary encoders which change color, and 1 for some "halo" light arround each panel).
I'm planning to have wireless program capability since getting to each node just to change a the color of a LED for example is quite a nuissance, not to mention getting to the ceiling ones controlling RGB LED strips.

The bootloader Joe created seems great since it does not require an external flash. Which would add cost and complexity to my wire rat's nest nodes.
I tried PMing Joe, but got no answer, maybe I'm not "old" enough on the forum to be able to send PM, the forum reported no error either.
So please Joe can you share your bootloader code with us? I want to use it as a starting point for developing my own and would be a great time saving.

Best regards,
Mircea.

PS. please excuse my English, not my mother's tongue
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on January 13, 2016, 04:33:19 AM
Hi Mircea,

sorry for the late reply.

I've periodically been thinking about how to release the bootloader. Doing so would obviously require work and so I only want to do it if it is really useful to many people. Currently it's heavily tied into my infrastructure and I don't think it's useful to just dump all my code on people (I did that to Tom and even he lost interest after the drop ;D).

I think it first requires some install server that people could use (either you just plug that into USB and program like an Arduino and it relays the program to the Moteino to be programmed, or it's a standalone server along the lines of the minimal gateway (esp8266 + rfm69w) we just discussed in another thread). Obviously the backend code could also just be integrated into Felix gateway but I find that pretty restrictive.

Then there would need to be a way for Moteino users to install it (bootstrap bootloader).

So overall: a lot of work. I currently have too many interesting other projects going on to make it a priority.

Joe
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: TomWS on January 13, 2016, 08:12:05 AM
(I did that to Tom and even he lost interest after the drop ;D).
Not true!  As you cite below, I have too many other interesting things going on to tackle this problem.  Especially when my code never needs to be updated!   :o
uh, JK!

Quote
So overall: a lot of work. I currently have too many interesting other projects going on to make it a priority.
Tom

PS: I predict that 'something' will happen 'sometime' this year on this matter.  It's far too interesting to disappear.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: executivul on January 13, 2016, 09:42:45 AM
Thank you Joe.
I'm a hardware engineer with some programming skills. I mean I can understand a piece of code unless it uses pointers to pointers referencing other pointers and tend to do things using simple for/while/if statements instead of #define code art.
Now I see that I can reserve 4K for the BL using fuses, which for me on 2560 is next to nothing. I would prefer reserving 16K for a bootloader instead of trying to optimize things, but 4K is the limit it seems.
Also all my nodes run wall powered for now, so power saving as long as it doesn't glow red hot it's nothing to bother for now.
I was contemplating having more networks, but it seems having a single one is simpler, still waiting for some parts from the far east to build more nodes and test rf crowding and rf separation for multiple networks running in the same space.
For now the plan is: the sketch sets the network parameters in EEPROM and the BL reads from there, tries to find a server/update, if available download and flash then boot. 2-3-5-10 second boot delay is not a problem either.

If it's possible just drop the code on me :) of course after removing all sensitive security data.
If I can use it I'll let you all guys know, though it might get a little less readable after I do my things. For example I took the clickEncoder library and because it didn't work with multiple encoders I stripped all the classes, functions, unnecessary variables and fancy coding, it's reduced now to just two functions and works for me, but for somebody having input active high it's harder to find and change the code than it was in it's original form. I'm a little code barbarian so to speak.

Best wishes.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: executivul on February 21, 2016, 05:07:06 PM
I'm still around :)
I've managed to compile an optiboot for my mega 2560 and I'm able to program via USB/UART up to 128KWords to it and it's all ok. I compile optiboot on OSX Capitan, avr-gcc from Arduino 1.6.7, make tools from Xcode (CrossPack failed me). Upload the BL via avrdude+AVRDragon. 622bytes  8)

Now the hard part: porting the RFM69.

Joe how did you manage to port the RFM69 library to the optiboot build environment? Did you strip all the classes and all the Arduino helpers (Serial.write, digitalWrite, etc) and re-made it in simple C or did you chose avr-g++ for optiboot and included everything from the Arduino environment? What size is your final bootloader?

Thanks.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on February 23, 2016, 02:30:13 AM
Yeah I stripped all those arduino helpers. Delay is also gone. Also I made all methods static with global instead of instance vars to avoid having to pass a this pointer around. It's still C++, but only in name since all methods are static, no virtuals etc. I'm polling instead of interrupts. And I optimized all around making it as small as possible.

My bootloader is exactly 4k. Every new piece I add I have to hunt somewhere to cut.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: executivul on February 23, 2016, 05:35:22 AM
By "Delay is also gone." you mean by any chance millis()? That's what I'm struggling with for now.
Polling instead of moving interrupts vectors to BL is the next on the list. The SPI interrupt seems the most troublesome.

Size is still a problem: on Mega 2560 I can have 8kB BL, but I also intend to use a bunch of 328p with just 4kB.
The simplest struct_send.ino compiles into 10kB if I use all the Arduino bells and whistles so that's a no no.

Thank you! Appreciate it.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on February 23, 2016, 07:00:27 AM
Yeah I meant millis. Since we only need to measure short periods no overflow is needed and you can easily just use one of the native timers instead.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: executivul on February 24, 2016, 07:01:52 AM
Look what I found: https://github.com/UKHASnet/ukhasnet-rfm69 (My rant about reinventing the wheel.)
It's a RFM69 + SPI library based on simple AVR calls, no Arduino "boiler plate" :)
Unfortunately I'm away and I can not compile it for the next few days...

Excuse me Felix if it's not appropriate to post this here.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: Felix on February 24, 2016, 09:09:55 AM
No problem, I am aware of UKHASnet, niec project, they have some nice info.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: executivul on March 02, 2016, 09:10:32 AM
By the way Joe, I liked so much your idea of CRCing the Flash and checking against server CRC that I decided to use it and get rid of all the versioning :)
But how do you know where to stop CRCing on the AVR?
I mean I can compute CRC from 0 to 0x1EFFF (in my case) that's all the flash except the BL, but I must then adjust the HEX CRC accordingly since the HEX does not contain all available addresses.
Or I can start from 0 and stop at 0xFF since unwritten flash is 0xFF but how long a string of 0xFF should be to be safely considered "past the end in no man's land"?
Or it's better that server sends the HEX's length in bytes and locally we CRC from 0 to the length received, assuming the new sketch is not identical and only longer?

Thanks.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 02, 2016, 09:59:39 AM
Yeah I think the CRC approach is really essential to making the bootloader resilient. Whatever happens - whether there's a transfer error, whether I change the hex file during update, whether there's a bootloader bug - in the end the app is started exactly if it got installed correctly. Otherwise I just try again.

I create two CRC's: one from 0x0000-0x6fff (the app) and one from 0x7000-0x7fff (the bootloader). On the server I just load the hex files into a similar sized region that I've previously filled with 0xff. Which leaves everything at 0xff unless it's present in the hex file. However I only transmit pages actually within the hex file if an update is necessary to minimize transfer time.

On the client you just need to erase all flash pages that aren't in the update prior to an update so that the CRC's match up again after install. Both sides just CRC the entire region then.

The reason I prefer to CRC the full regions is that I can transmit the CRC's with every bootloader request, including the first. No need to ask which pages to sum over before transmitting the CRC's. And having the CRC's in every call to the server allows the server to be completely stateless, serving multiple clients concurrently. I can restart the server during an update and everything proceeds just fine.

Joe
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 02, 2016, 10:14:00 AM
Quote
allows the server to be completely stateless

BTW, not only the server. The clients are also almost completely stateless. Partial install? Just restart, calculate new CRC and try again.

There's one bit of state which I do keep: whether the server has at one time approved what's currently installed on the client as not needing an update. I reset that flag when the server tells me I need to update. After installation is complete I just start another installation. If the previous install was successful the server will return that no update is needed and I set the flag before starting the app.

That flag allows one very convenient feature: a client can start its app if it's previously been approved by the server even if it can't reach the server during startup. That's possible because the flag allows a client to differentiate between an app that might merely be out of date and one that might be corrupted in absence of server communication.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: WhiteHare on March 02, 2016, 11:29:25 AM
If you're OTA updating the bootloader itself, is the new bootloader code saved in the space that's normally reserved for the sketch until the new bootloader code is validated and then used to overwrite the bootloader in its special part of flash memory?  That's clever.  Then on reboot the bootloader just calls for the sketch, as it apparently always does, and it just keeps calling until it gets a valid sketch. So, that's how you are guaranteed to avoid bricking a node.   Nice! 
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 02, 2016, 11:49:10 AM
Quote
If you're OTA updating the bootloader itself, is the new bootloader code saved in the space that's normally reserved for the sketch until the new bootloader code is validated and then used to overwrite the bootloader in its special part of flash memory?  That's clever.  Then on reboot the bootloader just calls for the sketch, as it apparently always does, and it just keeps calling until it gets it.

Exactly. Or more simply stated: a bootloader is replaced by an app that contains and installs the new bootloader (the "bootloader installer"). When the server sees a bootloader CRC which doesn't match it will consider the right app for the client to be the bootloader installer. Therefore the app CRC's don't match anymore leading to an app install of the bootloader installer. Once the bootloader installer is downloaded a restart happens and if the client's app CRC matches the bootloader installer CRC it gets executed and replaces the bootloader.

Another restart happens and the bootloader CRC's now match again so the right app for the client is it's previous app. Therefore the app CRC's don't match and the old app gets reinstalled.

Really pretty simple but it covers all cases and there is no chance for corruption or bricking other than installing a buggy bootloader. I'm running this stuff day in day out and it's so robust that I don't even put ISP headers on my boards.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: WhiteHare on March 02, 2016, 12:43:33 PM
Very elegant.  I really do hope this eventually becomes a standard way of doing things. 

To facilitate updating a sketch, I take it that the only thing that need be incorporated into the main sketch--or the main library used by the sketch--is a way for you to wirelessly force a reboot?  Or, instead, do you just have the node reboot every now and then and pick up the latest sketch that way?  Or I suppose it could be both ways, which would satisfy both the desire for immediate updates on a particular node (e.g. during a  rapid development cycle) as well as distributing less critical updates to all nodes.

Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 02, 2016, 01:16:03 PM
Quote
I take it that the only thing that need be incorporated into the main sketch--or the main library used by the sketch--is a way for you to wirelessly force a reboot?

Yeah exactly - that's one of the key benefits of this approach. You can install apps that know nothing about the deployment mechanism. Just assign the image to the node and plug it in.

I usually don't do anything automatically but manually decide which nodes I want to restart. For rapid development I just compile directly to the location where the gateway picks things up and send a reboot command to the node in question - all with one keystroke from emacs. I cross-compile from the mac so compilation is immediate. And then less then 4s later it's up on the node.

It's actually a fun way to see whether code changes caused binary changes: just compile and see if the node picks up a new version. Nice way to brush up on your C operator precedence skills ;-)
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 02, 2016, 01:20:34 PM
Quote
and send a reboot command to the node in question

Of course it helps that the nodes respond immediately thanks to listen mode - you see how it all fits together :-)
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: WhiteHare on March 02, 2016, 02:26:04 PM
I don't recall you ever explaining why you preferred 3 seconds rather than the 3 minutes you favored before, but I suppose that explains it: if you're wanting to update a node, 3 seconds is a tolerable incremental delay, whereas at 3 minutes it would be so irksome you might opt instead to walk over and wire-up the connection (well, back when you still had headers to interface with, that is!).  Is that how you arrived at 3 seconds?  i.e. it's the tradeoff that best satisfies both your patience and your battery life requirements? 
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 03, 2016, 11:14:41 AM
Quote
Is that how you arrived at 3 seconds?

I added listen mode to be able to interactively work with my nodes. I'll be on the command line and set parameters, send commands, read settings, update firmware etc. 3s was about the boundary where it still felt "live".
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: Felix on March 03, 2016, 03:50:41 PM
Hey Joe,
It's been a while, any chance you could release the wireless bootloader code?
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: TD22057 on March 03, 2016, 08:38:51 PM
Hey Joe,
It's been a while, any chance you could release the wireless bootloader code?

+1 - I'd love to see this as well.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: executivul on March 04, 2016, 01:19:55 AM
+1
(https://s-media-cache-ak0.pinimg.com/736x/79/39/a1/7939a15c9d7df2b075d91c65ecf49775.jpg)
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: WhiteHare on March 07, 2016, 02:40:04 PM
@joelucid

Given that your bootloader installation sketch presumably runs like an ordinary sketch, how is it able to write the new bootloader to the protected region of flash where the bootloader resides?
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 08, 2016, 02:45:41 AM
Quote
how is it able to write the new bootloader to the protected region of flash where the bootloader resides

It just uses the old bootloader and itself to write the protected regions. Takes a couple of tricks to make it work - I think I've explained earlier how I do it.

Quote
Hey Joe,
It's been a while, any chance you could release the wireless bootloader code?

As I've explained earlier I would only consider making the bootloader available as part of a solution that just works. In particular it needs a backend.

For example I could create a device that attaches to your wifi, can be fed with new versions of firmware which it will then cause clients to install. That way the bootloader backend would be independent of whatever other gateway one uses. Or my backend could be integrated into Felix gateway, I suppose. That would be a narrower solution but one that could boost adoption of the gateway.

But both options are significant work which competes with my other projects - and since the bootloader already works for me they never quite seem to make it to the front of my priority list. Although I have come at least as far as ordering a esp8266 dev board which is now sitting here to be tried out as install gateway. So I guess there's hope. Just keep pestering me  ;)
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: executivul on March 08, 2016, 04:37:28 AM
Quote
So I guess there's hope. Just keep pestering me  ;)

My bootloader is almost working using a modified WirelessHEX69 lib integrated in optiboot and Felix's python script. It's about 6KB and still unreliable, not to mention I do not use any of your speed improvements yet.
Yours seems much better so I would be more than happy to use it instead.
Please release it in however form you consider, I must modify it anyway to match my atmega2560 nodes so it won't be quite plug&play.

Vielen Dank!
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 14, 2016, 05:46:04 AM
Quote
It's about 6KB and still unreliable, not to mention I do not use any of your speed improvements yet.

Just a little progress report: I decided if I make this available I should optimize and clean it a little. Turns out there was still a LOT of dead wood around. Without coin cell support I'm at 2570 bytes now. With coin cell support it's 2912 bytes.

I've added what I think will become a pretty cool feature: every client gets assigned a worldwide unique id on first boot. Then the bootserver just assigns dynamic network node ids as clients boot up - sort of analogous to DHCP.

This will enable global node configuration later on - where I just configure my nodes once and they will feel at home with whatever gateway I connect them to. Be it one of several operating in a large area. Or just the case of my weekend home gw vs my home gw.

I've also started to add support for non 328p AVR's and have a Moteino Mega ordered from Felix so I can test that later on.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: executivul on March 14, 2016, 06:01:32 AM
Wow. Impressive work.
Do you still have the serial support?

Now I'm struggling with ICSP while having the RFM connected. I use 5V Arduinos so a level converter is needed, but the OE for the level converter is active high on the 3.3V side, so while I have a pulldown for the RFM not to interfere with the ICSP it seems I also must use another pin for OE of the level converter through a voltage divider... lots of extra wires and components I'm not sure I want, maybe I can get away with some 1K-2K resistors in series with the SPI lines. Should buy some more spare time for testing but can't find it available anywhere  :o

Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 14, 2016, 08:17:12 AM
Quote
Do you still have the serial support?

No - I'm trying to make it as small as possible.

Quote
Now I'm struggling with ICSP while having the RFM connected. I use 5V Arduinos so a level converter is needed, but the god damn OE for the level converter is active high on the 3.3V side, so while I have a pulldown for the RFM not to interfere with the ICSP it seems I also must use another pin for OE of the level converter through a voltage divider... lots of extra wires and components I'm not sure I want, maybe I can get away with some 1K-2K resistors in series with the SPI lines. Should buy some more spare time for testing but can't find it available anywhere  :o

Gosh that sounds complicated. Maybe convert the Arduino to 3.3V? I see no advantage in running a mixed setup. https://learn.adafruit.com/arduino-tips-tricks-and-techniques/3-3v-conversion
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: executivul on March 14, 2016, 06:04:16 PM
Thanks for the link. Interesting read, but unfortunately mega 2560 requires 4.5V for stable 16Mhz operation. I guess 3.3V it's just too low to be considered safe. Using 3.6-4V just to force the RFM a little over max and the mega a little under min also seems like a recipe for bad things.

I need the 16Mhz since there are quite a few things to take care of:
-RFM69
-OLED
-6-8 Rotary encoders
-6-8 Relays
-2x PCA9685
-DS18B20 (local in wall temperature for safety)
-SI7021
-DHT22 (already have bought them, then decided to go Si7021 for better Hum accuracy but still keeping them for comparison sake)
-Analog light sensor
-Analog sound sensor

I have a mean loop time of 15-17ms with spikes as long as 300ms when reading some of the sensors. I guess I have to optimize some of the libs used or change them to state machines, use more interrupts and/or prioritize things.

(http://i438.photobucket.com/albums/qq109/executivul/2016-03-12%2019.48.03.jpg)
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: WhiteHare on March 14, 2016, 06:30:37 PM
@executivul: Eventually loose wire connections are going to drive you mad, especially on some components (like the RFM69) that are so sensitive to slight voltage changes and noise. I do wish the diameter of the dupont male wire connectors was just slightly larger (or the female slightly smaller) so that the connections would be more snug. FWIW, I switched to soldering as much as I can as soon as I can, and it has been a lot less frustrating (though obviously more time consuming). 
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 14, 2016, 07:09:53 PM
I think you could run these sensors just fine at 8mhz using the internal oscillator if you continue having problems with the level converters. The sensor code just sits in delay and that takes the same time regardless of processor frequency.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: executivul on March 15, 2016, 07:01:00 AM
I like to test first on breadboard then go the soldering route. What you see there it's just the alpha state switch node prototype. I was curious how much power they draw, how hot the LM2596 gets, how do all parts work together. Looks promising until now.
After I receive my 3D printer tomorrow and get it assembled and running ok I can start building one or two "betas" of the future wall switches, all soldered using kynar and cat5 for power.

Joe I have no delays in my main loop(), all readings are done in a state-machine way since having a few variables for states is no problem, I use only 35K of flash and 2K of RAM. The only delays/locking loops are the ones used in libraries. Servicing the encoders takes quite a few lines of code for each and there are 8 of them, the Ada display library is a huge mess, about the DHT, 1Wire and I2C I have no idea since I didn't have the time to take a look at their code.
Still working on a state-machine RFM69 hack to get rid of those while() in tx and rx but that's another story.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: WhiteHare on March 16, 2016, 12:38:39 AM
I just now noticed that there was a notionally similar approach to OTA wireless programming back in 2012:  http://jeelabs.org/2012/10/31/meet-the-wireless-jeeboot/#comments and http://jeelabs.net/projects/jeeboot/wiki/Design_Notes

Unfortunately, it apparently never got developed much beyond the initial POC, and it seems to have more or less died on the vine.  What a pity!
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 16, 2016, 01:19:50 AM
Quote
I just now noticed that there was a notionally similar approach to OTA wireless programming back in 2012

Very interesting! I hadn't seen that. Anybody know why it died?
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: Felix on March 16, 2016, 08:39:02 AM
From what I gather it's actually very different technically than joe's method.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: WhiteHare on March 16, 2016, 10:32:01 AM
I doubt it's as refined.  It does have the virtue of being published, though, so for folks like executival who are chomping at the bit, maybe it makes sense to have a look and/or try it rather than just waiting indefinitely.  Interestingly, its bootloader is only about 2K in size.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 16, 2016, 12:36:05 PM
Quote
From what I gather it's actually very different technically than joe's method.

At least on the protocol level it's very different - given it takes a minute for an install to complete with his method. And I think that also explains the difference in code size. Otherwise there are many similarities. That is to be expected since we're solving the same problem.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: executivul on March 17, 2016, 10:24:15 AM
Thank you WhiteHare. I've searched a lot for other OTA bootloaders, found some NRF24 (nekromant's) but did not stumble upon this.
Now I'm working on the node code itself and maybe in the meantime Joe will share his solution. Hope dies last!
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 17, 2016, 11:28:44 AM
Quote
Hope dies last!

I'm working on it. If I change my plan I'll let you know.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: executivul on March 17, 2016, 12:21:08 PM
Thanks Joe! I mean it!
I won't be able to move in the new house until spring/summer next year. It's still under construction, building brick walls and roofs. In a few months I hope to start placing the wires and pipes, until then I must finish designing the switches so I can choose a in-wall box size for them. I've bought a 3d printer to help me build a frame to hold all the components inside the in-wall box. I must finish drawing the wiring schematics for 220V, 12V, Ethernet, TV, alarm. Some things are switched locally by small relays, like lights, while some are switched at the fuse box by large relays (big loads like washers). Radiators are individually controlled. Lots of things to consider. Then I must finish building all the nodes and I must test everything for reliability, WAF must be as close to 100% as possible and remaining in the dark/cold is not an option.
So until I move in the new house and start running with my laptop from node to node to change a sketch the wireless bootloader is not really needed.
Please take your time and release it whenever you consider.
Sorry for the off-topic and I promise not to bug you anymore about it.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: TD22057 on March 17, 2016, 05:37:22 PM
So I've been re-reading this thread and it seems to me like there are two separate features that joe has implemented:

1) Radio speed improvements.  Streaming mode, bit rates, and radio code w/o any waits.  This is pretty general purpose radio functionality and should be useful to any project I would think.

2) Automatic wireless programming.  From what I can see, this is a feature of the gateway which transmits code to the nodes if a new set of firmware is available for that node.

I was thinking more about 2) and it seems like that should be pretty straight forward to implement now (at a slower rate than joe's) by using the command line wireless programming tool that already exists.  Basically run a small script that looks for new firmware in some set location on the gateway and initiates a download when it shows up.  This isn't really different than now using wireless programming, just that the transmission starts from the gateway, not the user.  This doesn't require any new boot loader or listening modes to achieve - that makes be think I'm missing something about this feature though.  So do I have this right or is there something else going on that I'm not thinking about?
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: joelucid on March 17, 2016, 06:19:38 PM
Quote
So do I have this right or is there something else going on that I'm not thinking about?

The piece you might have missed is that the boot loader does it without the need for an external flash chip. That's one of the biggest benefits and enables much smaller or more fully featured designs.

Listen mode helps a bit since if allows you to trigger a reboot in an ultra low power node where you'd otherwise have to wait for it to phone home. But it's a pretty independent feature.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: TD22057 on March 17, 2016, 06:41:16 PM
The piece you might have missed is that the boot loader does it without the need for an external flash chip. That's one of the biggest benefits and enables much smaller or more fully featured designs.

Thanks - I felt like I must have been missing something there. 
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: brolly759 on May 11, 2017, 12:41:32 AM
Joe, I used xbeeBoot code in one application already, is this pretty much the same concept but with RF69 chips?

https://github.com/davidsainty/xbeeboot
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: zingbat on January 26, 2018, 04:55:22 AM
Hey Joe,

Would you be willing to release the bootloader? I’m very interested in unbrickable, remotely programmed nodes for locations that will be hard to access. I’m happy to modify the bootloader and de-couple it from the protocol/throughput enhancements you made, unless you’d be willing to share that too.

Thanks!
Jeff
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: Felix on January 26, 2018, 10:48:27 AM
Hey Joe,

Would you be willing to release the bootloader? I’m very interested in unbrickable, remotely programmed nodes for locations that will be hard to access. I’m happy to modify the bootloader and de-couple it from the protocol/throughput enhancements you made, unless you’d be willing to share that too.

Thanks!
Jeff
Last time people asked, Joe talked about how it works but it was closed source.
Did you look into Dualoptiboot (https://github.com/LowPowerLab/Dualoptiboot) and wireless programming (https://lowpowerlab.com/guide/moteino/wireless-programming/)?
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: zingbat on January 27, 2018, 12:07:34 AM
Thanks, I have been using Dualoptiboot and have experimented with the current OTA approach.

Two other factors I didn't mention originally that steer me towards a solution resembling Joe's:
1. I prefer not to have flash on every sensor node, given the power budget/assembly time/cost
2. I will be epoxy potting the nodes in order to weatherproof/condensation proof them for outdoor usage. This will render all of the programming pins inaccessible after initial configuration, so I'm looking for a system where an inadvertent programming error will be recoverable.

Jeff
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: LukaQ on January 27, 2018, 01:59:56 AM
You could make those few pins be accessible after potting, I think you would still have waterproof item and also be able to reprogram it always
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: Felix on January 27, 2018, 07:58:57 PM
1. I prefer not to have flash on every sensor node, given the power budget/assembly time/cost
2. I will be epoxy potting the nodes in order to weatherproof/condensation proof them for outdoor usage. This will render all of the programming pins inaccessible after initial configuration, so I'm looking for a system where an inadvertent programming error will be recoverable.
IMO both are subjective tradeoffs. I went with the external FLASH approach specifically to avoid any potential risks. I really thought hard and long about it when I designed them into the Moteino. I knew that if OTA didn't fail on me, it would fail on someone else. I didn't need an extra part in the BOM for no reason. Plus that part required me to buy a very expensive vibe feeder in my pick and place back when I had the machine.
If your sketch is less than half of the available flash memory in the MCU, then yes - you would not need external flash at all, you could always use the other half. But in my world - overwriting yourself with no insurance of backup is a risk I can't take, and I never epoxied any of my nodes I can't easily reach.
Title: Re: Wireless Boot in 2.5 Seconds / High-Speed Datastreams
Post by: zingbat on January 31, 2018, 04:41:31 AM
Thanks, and definitely agreed that for the standard case, the current OTA approach is safer/better. I think I have somewhat unusual requirements given that I'll be physically occluding the programming pins on the Moteino.