LowPowerLab Forum

Please vote.

Ill say its easy cause Ive done it so many times, but that would tilt the statistics unfairly.
On my million TODO list somewhere is the "make a really nice GUI for wireless programming", so one day :)
BUT .. there is already one such GUI created by luisr320 (https://lowpowerlab.com/forum/index.php/topic,967.0.html), quite functional. I was thinking of more bells and whistles :P

Quote from: WhiteHare on October 12, 2015, 03:29:16 AM
Please vote.

Can we edit the poll to add as a second choice:
"Yes, but only when necessary because it is a strain on battery operated Motes."

Also, the poll doesn't distinguish between OTA methods so you need to be careful with your conclusions.  Some methods are easier/more efficient than others.

Finally, if this poll is an attempt to determine need for an ESP OTA solution, I'm failing how you'd be able to make that specific conclusion from these questions.

Tom

Quote from: TomWS on October 12, 2015, 08:32:10 AM

Quote from: WhiteHare on October 12, 2015, 03:29:16 AM
Please vote.

Can we edit the poll to add as a second choice:
"Yes, but only when necessary because it is a strain on battery operated Motes."

Tom

Done!

I thought a poll would be an interesting way to compare notes.  I'm not striving for any particular conclusion.  Presently the sample size is so small I definitely can't infer much of anything.  I thought the poll would at least show who answered what, but it doesn't, so it's of diminished value.  Most people on this website are guests who can't vote  and probably wouldn't even if they could.  So, meh, like a lot of polls, it's not going to prove anything.

This guy (https://www.youtube.com/watch?v=3Zcg8hwDSXI) appears to be using an esp8266 to upload sketches, but the details seem rather sparse.  An esp8266 costs $2.50, delivered.  If I can wack one in for $2.50 and make uploading go faster and easier and save myself headache, then that's a win in my book.  Of course, if it's easy to do without an ESP8266, then that's a win too.  I just don't want to open another can of worms, that's all.

Quote from: WhiteHare on October 12, 2015, 12:52:29 PM
<snip>
An esp8266 costs $2.50, delivered.  If I can wack one in for $2.50 and make uploading go faster and easier and save myself headache, then that's a win in my book.  Of course, if it's easy to do without an ESP8266, then that's a win too.  I just don't want to open another can of worms, that's all.

I say, "Lead the charge!".  If you see value in it then others might as well.  Make a working prototype and post the results.  PS: Try to be server/gateway agnostic if you can, ie, HTTP requests might even work with Google supplying the images!

Tom

I just went to vote but the poll is closed.  Did you do limit the time frame on purpose?

It should work now.  For some reason it locked the poll when I added your extra choice.  I didn't realize that until you mentioned it.  I just now clicked unlock, so hopefully it's all good now.

I've thought about how to make ota installs easy in the Tino context. The idea I've come up with would also work with Felix approach: Take a Moteino usb and write an app that emulates optiboot at the USB port, downloads the app to external flash when the arduino IDE "programs" it and then uses the radio to do OTA programming of a node. If more than one node is in use provide a little gui to tell the Moteino USB which node to reprogram.

The advantage is that this makes OTA programming completely transparent to users of the Arduino IDE.

Quote from: joelucid on October 13, 2015, 02:21:23 AM
I've thought about how to make ota installs easy in the Tino context. The idea I've come up with would also work with Felix approach: Take a Moteino usb and write an app that emulates optiboot at the USB port, downloads the app to external flash when the arduino IDE "programs" it and then uses the radio to do OTA programming of a node. If more than one node is in use provide a little gui to tell the Moteino USB which node to reprogram.

The advantage is that this makes OTA programming completely transparent to users of the Arduino IDE.

It would certainly make a nifty little product for LowPowerLabs... Hint, Hint, Felix.

There is one challenge, however, the MoteinoUSB would have to be able to handle multiple serial rates on its 'optiboot emulation' since the download speed changes based on Board settings (eg, 8MHz Moteinos download at 57600).

Tom

Tom - I replied to your new dedicated thread (https://lowpowerlab.com/forum/index.php/topic,1350.0.html) for this. Yes I'm really liking what I read here.
I do think it's an advanced topic that needs to be engineered such that a non-advanced user will be able to use it. We can continue the discussion in that thread.

My problem with Felix's current solution is its code size. It's pretty big for something that has all other componens readily available in a sketch anyway, like the radio code and SPI flash library. IIRC it's around 4-5k flashed. Otherwise a $1 flash device is a shallow price to pay for OTA caps and the bootloader is still only half k. If Joe's current solution is not significantly smaller - it seems he needs a 4k one, which always bites into available pgm space, then it might not be the best alternative. Actually I was thinking about it a lot - it's more elegant, less parts needed and thus potentially lower power consumption, cheaper. Dunno...

However. The arduino IDE's flashing tool chain can be replaced by config changes in the IDE. By default for Uno-like devices and many others, avrdude gets called. If we have a moteino sitting on the serial port with the proper code to do remote flashes, and a small app/gui to select the target mote to flash, or maybe a specially crafted small data block in code/data to ID it through the .hex, remote flashing becomes a trivial matter for anyone using the basic IDE (or configuration-compatible ones, like Visual Micro). That, in my opinion, could give a boost to OTA programming (and one more mote to sell for Felix for OTA ;) )

If anyone sees point in going that route, I'm willing to contribute or start working on it at some point. I think I could manage Java patch on the IDE, should it become necessay (though I don't think it is).

Quote from: kobuki on October 15, 2015, 08:37:12 PM
My problem with Felix's current solution is its code size. It's pretty big for something that has all other componens readily available in a sketch anyway, like the radio code and SPI flash library. IIRC it's around 4-5k flashed.

Just curious: if the radio and SPI code, or a near equivalent, were folded into the bootloader, would the code still be accessible from within the regular application program space?  Or would it require some duplication because of it residing in a protected region of flash?  4-5K is one thing (unlikely to be a show stopper for me, but maybe for you), but 8-10K would obviously run a bigger risk of running out of space.  If push came to shove, would Moteino Mega be an acceptable fallback for you, or would you (I'm guessing) deem OTA Programming not worth it if that were the trade-off?  Just for the added convenience, I, for one, could imagine myself using Moteino Mega's with OTA programming for at least preliminary development work, even if I were compelled to shift to ordinary non-OTA Moteino for deployment. 

QuoteIf Joe's current solution is not significantly smaller - it seems he needs a 4k one, which always bites into available pgm space, then it might not be the best alternative. Actually I was thinking about it a lot - it's more elegant, less parts needed and thus potentially lower power consumption, cheaper. Dunno...

That's exactly why Tom's idea of using a basic default bootloader with the capability of upgrading it to ota is so appealing. One solution doesn't need to fit all.

QuoteIf we have a moteino sitting on the serial port with the proper code to do remote flashes, and a small app/gui to select the target mote to flash, or maybe a specially crafted small data block in code/data to ID it through the .hex, rem

They call it the "nifty device"  :D - btw first idea was to just have it emulate the avrdude protocol so it just works everywhere. But you're right we could also and maybe optionally implement our own protocol given how poorly the standard protocol performs esp. via USB.

Quoteif the radio and SPI code, or a near equivalent, were folded into the bootloader, would the code still be accessible from within the regular application program space?  Or would it require some duplication because of it residing in a protected region of flash?

Sharing is possible. I haven't done it since I have massively modified the radio lib to support multiple profiles, burst mode, polling instead of irqs and reduce size etc. I also think you'd want to modify the bootloader as rarely as possible so keeping things separate is probably the right approach.

Still it's possible and could be done in flash constrained environments. Might save 1k in the end (most of the code is probably bootloader logic / battery mgmt if you optimize the radio code).

With a small bootloader such as the DualOptiBoot, I have the choice to use or not to use OTA (given I have a flash soldered, of course). You can't reasonably go lower than that with a bootloader. The OTA uploader code takes space either in the bootloader restricted area or in user space, but it does take space somewhere. IMO using a library through the bootloader area is very cumbersome, part-specific, and is just a big hack, not for general use. I don't like the idea -- and I'd like to stress out here that it's just my own opinion. Adding a simple hook for self-update might be on the borderline.

Anyway I just wanted a simpler upload procedure for OTA so I will just go the route I mentioned in my previous post. I'll report back after I'll have got it working. Don't know when, but I will eventually get to it later. Unless someone else does it, like Felix who knows his way around his products and code a lot better than anyone else.

Adding my 2 cents to the discussion and as an insight ... the idea of the Moteino OTA wireless programming and the choice of using a FLASH and slightly modified (just an extra 512 bytes more) DualOptiboot was long and hard pondered upon when I developed it.

I weighed all the choices available at that time (not many actually) and thought of what is the best all around method that will be flexible, relatively easy to understand and use, only require the extra chip (which boosts confidence of a non interrupted or corrupted image), requires little code (just an extra include and a function call) .. all to make wireless programming easy to integrate in an existing sketch. There are of course many ways to do it, all with pros and cons.

If I were to do it again, I would probably do it the same still. It has proven rock solid with no fails nor frustration, I use it regularly myself and its just awesome to be able to update a switchmote in the wall without disturbing the rest of the node network and without having to disconnect anything .. it just works when all steps are followed and the right HEX sketch is uploaded.

I admire the effort, discussion and great ideas proposed by people in the forum and I'm all for it. Keep it coming, and thanks for sharing all this great stuff here guys, it's certainly very valuable. But I hope people see where I'm coming from.

Quote from: joelucid on October 16, 2015, 03:20:56 AM

Quoteif the radio and SPI code, or a near equivalent, were folded into the bootloader, would the code still be accessible from within the regular application program space?  Or would it require some duplication because of it residing in a protected region of flash?

Sharing is possible.

Well, then, if sharing is possible, I would argue that the radio and SPI code should go in the bootloader, because then OTA Programming could be used as the default, straight out of the box.  Unless you are planning to not load the radio and SPI code at all (in which case, why are you using a Moteino at all?), I don't see how it is that you're "saving" flash space by putting it outside the bootloader.  When you add it all up, wouldn't the total amount of flash that's occupied be the same either way? 

Quote from: WhiteHare on October 16, 2015, 12:55:05 PM
Well, then, if sharing is possible, I would argue that the radio and SPI code should go in the bootloader, because then OTA Programming could be used as the default, straight out of the box.  Unless you are planning to not load the radio and SPI code at all (in which case, why are you using a Moteino at all?), I don't see how it is that you're "saving" flash space by putting it outside the bootloader.  When you add it all up, wouldn't the total amount of flash that's occupied be the same either way?

Partly this is why I said I don't really like the idea -- the radio and flash/SPI code needs space either way. But add to that the fact that in the available boot loader space you still need to code things on a minimal footprint, and then you don't have the capabilities and features of the full-fledged libs. And you need quirks to access them. If we had like 64 kB or ram or more, this would be a no-brainer and Moteinos should ship with a... hm, well, TripleOptiBoot ;)

Quote from: kobuki on October 16, 2015, 01:01:59 PM

Quote from: WhiteHare on October 16, 2015, 12:55:05 PM
Well, then, if sharing is possible, I would argue that the radio and SPI code should go in the bootloader, because then OTA Programming could be used as the default, straight out of the box.  Unless you are planning to not load the radio and SPI code at all (in which case, why are you using a Moteino at all?), I don't see how it is that you're "saving" flash space by putting it outside the bootloader.  When you add it all up, wouldn't the total amount of flash that's occupied be the same either way?

Partly this is why I said I don't really like the idea -- the radio and flash/SPI code needs space either way. But add to that the fact that in the available boot loader space you still need to code things on a minimal footprint, and then you don't have the capabilities and features of the full-fledged libs. And you need quirks to access them. If we had like 64 kB or ram or more, this would be a no-brainer and Moteinos should ship with a... hm, well, TripleOptiBoot ;)

I don't know about footprint requirements, so I don't know enough to evaluate that, but I'm pretty sure you wouldn't have to put the entire radio library into the bootloader, just the part that's needed for the OTA Programming.  So, rather than a monolithic library, you could divide it into two libraries: 1. the bare minimum code needed for OTA Programming and which goes into the bootloader, and 2. all the rest of it, which needn't go into the bootloader, if there are footprint limitations.

How big a bootloader footprint is possible?  Maybe that's the critical question.  If it's 8K, then it should be easy, right?  I mean, the original arduino bootloader was what, 4k?  Now optiboot is 512bytes.  If 4k were the limit (and I don't know that it is), could spi and the essential radio code be fit into 3.5K? 

Quote from: WhiteHare on October 16, 2015, 01:34:29 PM

Quote from: kobuki on October 16, 2015, 01:24:27 PM

Quote from: WhiteHare on October 16, 2015, 01:15:36 PM
I don't know about footprint requirements, so I don't know enough to evaluate that,

How large is your own, non-flash OTA bootloader?

I'm not sure what you're asking.

Wah, sorry, I didn't pay my full attention to the forum ;) I wanted to ask Joe about that, sorry. I deleted my post.

If the entire bootloader needs to reside in the NWWR section, then from Table 26-14 of the 328p datasheet, the maximum size is just 2K.  :(

Maximum bootloader size is 4K (that's also the size of mine) but as I've said previously nothing can stop you from calling from the bootloader into normal flash so as long as the jump tables are generated correctly you could add code below the bootloader which it would use.

That said I think the bootloader should really be a fairly static piece of code. I wouldnt want to have much of the regular libraries in it so that they can be updated without changing the bootloader. I think the 4K ota bootloader is a sweet spot that works well in all but the most extreme cases.

Bottom line: ota updates are hugely attractive. Felix method and the bootloader method both work each with their own pros and cons. In some cases only the bootloader works:

When you don't have space for the flash
When you need special code to enable coin cells
If you have a habit of uploading broken hex files :) - happens to me all the time.
If you need the option of ota bootloader changes.

We have a potential mechanism for replacing the bootloader. But all the discussion is moot unless there is actually a ota bootloader that's publicly available. Mine works but is tightly coupled with my environment plus I don't yet have the nifty device. So for now we're stuck with a vision of what could be done bootloader wise  - and Felix existing, stable and reliable method.