Felix et al,
I am writing some custom code to write hex data to flash for the purposes of wireless programming the Moteino, the question I have is that when writing the hex data to flash do I write the "3A" byte or colon that is found at the start of each line in the original hex file ?
BR Peter.
ie should it look like this...
....
3A.10.7.90.0.EE.1F.FF.1F.A2.17.B3.7.E4.7.F5.7.20.F0.A2.1B.7.
3A.10.7.A0.0.B3.B.E4.B.F5.B.66.1F.77.1F.88.1F.99.1F.1A.94.74.
3A.10.7.B0.0.69.F7.60.95.70.95.80.95.90.95.9B.1.AC.1.BD.1.9E.
3A.10.7.C0.0.CF.1.8.95.AA.1B.BB.1B.51.E1.7.C0.AA.1F.BB.1F.85.
3A.10.7.D0.0.A6.17.B7.7.10.F0.A6.1B.B7.B.88.1F.99.1F.5A.95.CD.
3A.10.7.E0.0.A9.F7.80.95.90.95.BC.1.CD.1.8.95.EE.F.FF.1F.EC.
3A.C.7.F0.0.5.90.F4.91.E0.2D.9.94.F8.94.FF.CF.DF.
3A.10.7.FC.0.0.0.C6.1.80.0.48.65.6C.6C.6F.20.48.61.0.0.E9.
3A.10.8.C.0.0.0.0.C4.1.40.3.44.1.5F.1.51.1.A2.1.0.3A.
3A.0.0.0.1.FF.FF
No, you only need the header that contains the colons, once the image starts it's uninterrupted. Here's the format:
FLXIMG:NN:_________NN image bytes__________________
Felix,
This is node based code, I am not using your libs for this, what is the format as it is actually stored in FLASH memory, from what I could determine you weren't writing the FLX header and sequence number to FLASH memory, only the HEX data....
P.
Felix,
My bad, I misunderstood your last post !
Thanks for the clarification !
BR P.
Ok no problem. And yes the data is written to flash just like that, with the 10 byte fixed format header where the NN bytes represent a SHORT == to the actual size of the image.
The bootloader checks for the presence of this header, reads the NN number and then writes the following NN bytes into the internal atmega flash, then erases the block from the external FLASH.
Got ya, had another look at the code and along with your comments it all makes sense now.... thank ya sir !
Interesting find... Arduino IDE reports the sketch size as around 26,000 bytes, and the image can be written to the moteino fie using Arduino IDE, however the resulting hex file has around 1640 lines and at 21 bytes per line that is giving an image size of over 34,000 bytes.....
Has anyone seen this before ?
P.
Please see the intel HEX format spec. Each byte is written in ASCII which means it takes 2 times more space.
https://en.wikipedia.org/wiki/Intel_HEX
Also everything is big endian so you have some processing to do before you dump that into the FLASH for DualOptiboot to pick up. Everything is done as an example for you in the WirelessHEX69 lib:
https://github.com/LowPowerLab/WirelessProgramming
Hello everyone,
First post here :-)
I've worked quite a bit with JeeNodes and I'm very interested in this wireless programming. Right now I'm trying to understand what's going on in all these different parts. Here some notes I made. Maybe these are of some use for someone else.
Format of data stored on external flash chip:
"FLXIMG:SS:xxxxx"
SS is size of following firmware in bytes stored as uint16_t
xx is the firmware itself, where the Bytes are in Big Endian (flipped in bootloader before writing to IC)
Bootloader checks for "FLX___:__:" (_ is not checked) and checks if the size of the new firmware is an even number of
bytes. If this is successful, the new firmware is written and the first 32k or 64k (depending on firmware size) on the
external flash are cleared.
There is a small issue in the documentation of the boot loader. https://github.com/LowPowerLab/DualOptiboot/blob/master/Optiboot.c on line 44 the size is labeled as being 4 bytes size instead of 2 bytes.
On-Air Firmware Update between Server (S) and Client (C).
S>C indicates a message being sent from Server to Client
// Handshake, initiated by server
S>C: "FLX?"
C>S: "FLX?OK"
C: erases first 32k on external Flash and Writes: ,,FLXIMG:__:" at the beginning (__ is two empty bytes (\0))
// Firmware transfer
S>C: "FLX:++++:***" : (++++ is 1-4 Bytes of Sequence Number as ASCII Number (Base 10) / *** is flash data, limited only by maximum RF packet size)
C>S: "FLX:++++:OK" : (++++ is the same as received sequence number)
... repeated until complete firmware is transmitted ...
Sequence number is always incremented by 1, not the length of the transmitted payload.
// Finishing
S>C: "FLX?EOF" : Client checks size and if OK writes size as uint16_t to Bytes 7-8 in Flash.
C>S: "FLX?OK" : 16ms after sending Clients resets itself using watchdog.
Lines 29-32 in WirelessHEX69.cpp should be useless as there should have been a watchdog reset before the server times out.
Lines 95,96: You're checking if there is an empty sequence number? I don't see how this should ever happen but maybe some additional checks aren't a bad idea.
Is there a reason why you're sending the sequence numbers as ASCII number? This one is not obvious to me as it should be easier to just use a uint16_t binary representation. Always the same size, less bytes to transmit and bigger range. This also simplifies some code where you extract the variable-length sequence number out of the packet.
A great piece of software you've written here. Quite easy but so powerful. Thanks for sharing!
Welcome tht and thanks for sharing your notes and feedback.
I fixed the bytes documentation on line 44, thanks for pointing that.
The empty sequence and EOF resend are there to avoid issues and make it more robust in case other types of packets are received that don't match the format. Lost EOFs do happen sometimes, though rarely. So we don't want the target to fail right at the end in case it did not receive the EOF.
The sequence is sent as ASCII because at the cost of 2 extra bytes you can visually look at the data and understand what's going on right away without doing calculations.
Hi Felix,
I wanted to loop back on your comments below, I was aware that the hex file generated is considerably larger, each line consists of 43 characters or 42 bytes of code data if we exclude the colon, when this is transmitted to the receiving node I use the bytefromhex function to convert each "ascii hex pair" to a single byte of data which represents the original hex value, hence for each line contained in the original ascii hex file we end up writing 21 bytes of actual data to flash.
Looking at your code I assume this is normal enough, the point that I am making is that the same sketch I am writing with Arduino IDE which is listed as circa 28k when uploading via the Arduino IDE ends up being in excess of 35k when I wireless upload the hex file and write it to flash.
Is this expected behavior ?
I am assuming that it is a hard fact that we are going to write 21 byes of data to flash for each line in the original hex file as I don't think we are doing anything more complicated that converting each ascii pair to a hex byte. I can only assume that the IDE is doing some sort of optimization on the code before uploading that we don't get with the raw hex file, is this plausible ?
BR Peter.
Quote from: Felix on August 30, 2014, 09:15:24 PM
Please see the intel HEX format spec. Each byte is written in ASCII which means it takes 2 times more space.
https://en.wikipedia.org/wiki/Intel_HEX
Also everything is big endian so you have some processing to do before you dump that into the FLASH for DualOptiboot to pick up. Everything is done as an example for you in the WirelessHEX69 lib:
https://github.com/LowPowerLab/WirelessProgramming
Peter,
Each line is 16 bytes of data not 21. The extra bytes are the HEX header and the last byte is CRC. Again, please see the intel HEX format:
http://en.wikipedia.org/wiki/Intel_HEX
If you get a 28K sketch the TOTAL number of bytes written is : 28K (sketch) + 10bytes (header).
You will never get more than 32K because the bootloader is 1K.
Ahaa!
So I should be stripping the header and CRC from each line of the hex file before writing the remainder of the data to flash ?
That wasn't obvious to me from your sample code, but that's my bad. Are you stripping these bytes in the phython script before sending across serial or at the gateway mote before wireless transmission. I don't think you are stripping them during the flash write stage on the target mote...
BR Peter.
Quote from: Felix on September 04, 2014, 08:14:05 AM
Peter,
Each line is 16 bytes of data not 21. The extra bytes are the HEX header and the last byte is CRC. Again, please see the intel HEX format:
http://en.wikipedia.org/wiki/Intel_HEX
If you get a 28K sketch the TOTAL number of bytes written is : 28K (sketch) + 10bytes (header).
You will never get more than 32K because the bootloader is 1K.
The header and CRC are validated and processed at the programmer Moteino (or gateway if you want to refer to it that way), in the HandleSerialHEXData() function:
https://github.com/LowPowerLab/WirelessProgramming/blob/master/WirelessHEX69/WirelessHEX69.cpp
As a side note, I always use a "wireless programmer" Moteino dedicated for that purpose. I don't use the main gateway of the network since I don't want to interfere with the normal operation of the network.
Felix,
Thanks, it all makes perfect sense now ! Will have to tinker with my code a little further....
BR Peter.
Quote from: Felix on September 04, 2014, 08:47:17 AM
The header and CRC are validated and processed at the programmer Moteino (or gateway if you want to refer to it that way), in the HandleSerialHEXData() function:
https://github.com/LowPowerLab/WirelessProgramming/blob/master/WirelessHEX69/WirelessHEX69.cpp
As a side note, I always use a "wireless programmer" Moteino dedicated for that purpose. I don't use the main gateway of the network since I don't want to interfere with the normal operation of the network.