Wireless Programing while being able to use the rest of the SPIFLash

[jarrods]

Hello Community,
I have been actively working on a couple of projects and need to get some basic functionality setup to allow it to be scaleable. I just got my first set of Moteino's right before christmas (and 2 more on christmas day for personal use :-) and have been having a blast with them. I am looking to setup some predefined addresses (hopefully at the beginning of the address range) in the SPIFlash that I can use to store configuration settings. for example: Node ID, Network ID, the start / stop address for logging along with a few others. This way I can have a "generic" Sketch with no hard coded values. and use to to flash all my node even if there are in different networks or have different flash chips. When it comes online for the first time if the values from flash are invalid it will wait for programming from a Gateway and use a Temp Network (single network ID assigned to Initializing Nodes) and Node ID (randomized to limit collisions). Then it will wait to receive an identity / configuration from the gateway and reset, switching to that ID and config.

I am also adding the ability to the Python Script and Gateway sketch the ability for the Gateway to be flashed to SPIFlash via serial (the computer it will be connected to won't have the arduino environment installed.) and be able to change the network and node ID via Serial.

So here are my questions:

First is about about how the SPIflashed is used for programing wirelessly. I have the wireless programming working, it is great!!! Thanks Felix (and KanyonKris) for making it easy to do with multiple nodes. While looking through the WirelessHEX69.cpp I noticed that the HandleWirelessHEXData that it start flashing after block 10 thereby skipping the first 10 blocks(bytes). Is there a reason for that? are they used anywhere?

Second: would there be an negative impact if I were to change every reference in all the libraries to "blockErase32K(0)" to "blockErase32K(10)" so as to skip the first 10 blocks? what about the "FLASH_command(SPIFLASH_BLOCKERASE_32K, 1);" in the Optiboot? I am assuming I would have to change that as well?

Third: Also looking in the Optiboot.c I can see that the EEPROM is not enabled in the bootloader (lines 87 and 141 in comments, did not actually go through all the code) so i am guessing i can't store the "Configuration Settings" there

Forth: Am I over-complicating this or is there a simpler way? I would like to avoid changing the bootloader but also want to be able to support different flash chips in the future. Should I just give up on that and use the beginning of the second 32K block (address 32768 right?) on the SPIFlash?

Thanks all!

[KanyonKris]

Forum member "john k2ox" used your third method. I like it because using EEPROM keeps the configuration data separate from the SPI Flash used for wireless programming  - http://lowpowerlab.com/forum/index.php/topic,213.msg996.html#msg996
He didn't mention having to change the boot code to allow using the EEPROM of the ATmega. John offered to share his code but hasn't posted it yet. There is a library (EEPROM.h) for using the EEPROM. This example may be helpful - http://playground.arduino.cc/Code/EEPROMLoadAndSaveSettings

I like your idea of having the gateway keep a list of assigned IDs and give out new IDs when requested by a new node. Certainly doable. But for my uses I'm OK setting up the new node by hand (ie giving it an ID).

Your idea for reprogramming the gateway without the Arduino IDE is also good. Just use the existing functions for writing a hex file to the SPI Flash and reflashing.

Furthermore, Felix and I have been discussing some changes to the code to support both wireless programming and low power modes - http://lowpowerlab.com/forum/index.php/topic,241.0.html

[jarrods]

Thanks... It is funny you mentioned that cause I just finished testing that EEPROM library and it works just fine. So I am taking that route. I like the low power idea but not for the power saving, but because it frees up the serial. when your programing a node you cannot receive data.

I am looking at using and Inductive charger built into the case to recharge a 850mah LiPo and a small solar cell for the outdoor notes (will be a while before i build those however).

Thanks for the advice!

[Felix]

Hello Community,
I have been actively working on a couple of projects and need to get some basic functionality setup to allow it to be scaleable. I just got my first set of Moteino's right before christmas (and 2 more on christmas day for personal use :-) and have been having a blast with them. I am looking to setup some predefined addresses (hopefully at the beginning of the address range) in the SPIFlash that I can use to store configuration settings. for example: Node ID, Network ID, the start / stop address for logging along with a few others. This way I can have a "generic" Sketch with no hard coded values. and use to to flash all my node even if there are in different networks or have different flash chips. When it comes online for the first time if the values from flash are invalid it will wait for programming from a Gateway and use a Temp Network (single network ID assigned to Initializing Nodes) and Node ID (randomized to limit collisions). Then it will wait to receive an identity / configuration from the gateway and reset, switching to that ID and config.

I am also adding the ability to the Python Script and Gateway sketch the ability for the Gateway to be flashed to SPIFlash via serial (the computer it will be connected to won't have the arduino environment installed.) and be able to change the network and node ID via Serial.

So here are my questions:

First is about about how the SPIflashed is used for programing wirelessly. I have the wireless programming working, it is great!!! Thanks Felix (and KanyonKris) for making it easy to do with multiple nodes. While looking through the WirelessHEX69.cpp I noticed that the HandleWirelessHEXData that it start flashing after block 10 thereby skipping the first 10 blocks(bytes). Is there a reason for that? are they used anywhere?

Second: would there be an negative impact if I were to change every reference in all the libraries to "blockErase32K(0)" to "blockErase32K(10)" so as to skip the first 10 blocks? what about the "FLASH_command(SPIFLASH_BLOCKERASE_32K, 1);" in the Optiboot? I am assuming I would have to change that as well?

Third: Also looking in the Optiboot.c I can see that the EEPROM is not enabled in the bootloader (lines 87 and 141 in comments, did not actually go through all the code) so i am guessing i can't store the "Configuration Settings" there

Forth: Am I over-complicating this or is there a simpler way? I would like to avoid changing the bootloader but also want to be able to support different flash chips in the future. Should I just give up on that and use the beginning of the second 32K block (address 32768 right?) on the SPIFlash?

Jarrods - all great ideas, some of which have been on my mind for a while, especially allowing dynamic ID allocation. This is more or less like DHCP over RF
For that to really be effective and consistent, it would be great if a fixed unique ID could be built in each Moteino. A sort of MAC if you will. That can be done with a dedicated ROM chip, like even the dallas temp sensors have it, or a SPI device specifically for this purpose. However these parts are a few bucks, sounds like a lot, and you won't have a temp sensor on every Moteino just to get a unique ID. So the solution is to initially place a random unique ID somewhere (EEPROM, SPIFlash). If I make progress here I will blog about it but this is a good start I think. The idea is to have a way to assign the same ID to the same node after a power failure or a reset. Meaning ... I want my garage or my sump pump sensor to have the same ID for logging purposes, or if I have N temp sensors, I want to be able to know that node ABC is in the attic and DEF is in the basement, things like that.

About your SPIFlash questions...

1)
SPIFlash uses the first 32K block for wireless programming so you should stay out of that because it gets erased and rewritten. The first 10 bytes are skipped till the very end of the writing of the HEX because those bytes are used for signature and HEX byte count (which is unknown until the whole HEX flash image is transmitted). So in the first 32K block, only the end will be left empty if the sketch does not take the whole 32K. In fact the Dualoptiboot bootloader takes 1K, so effectively only 31K+10bytes max will ever be used if the sketch is maxed out. BUT because of the SPIFlash architecture it's easy to play with 32K blocks, but not so easy with individual bytes (can write a single byte but can't erase a single byte).

2,3)
The bootloader is just that .. a program that handles an incoming HEX image over serial. You should not try to add any program specific logic in there. My changes to make Dualoptiboot are an exception, so that the image could also be loaded from the SPIFlash chip. The moment you add .h files and code to the bootloader it will quickly increase in size and it's very hard to make very compact code. My Dualoptiboot changes took another 500 bytes to check/read and flash the image from the SPIFlash memory.

4) Sounds like you're already going to use EEPROM. That's a good idea. If EEPROM was 32K I would have used that for the wireless programming. But it's only 1K. So that's way less, but enough to keep basic config stuff in there. I believe Jeelabs took this approach as well. I would recommend this also if you want to keep a static ID in there.

[jarrods]

Thanks Kris and Felix for the feedback!

Over the last week or so I created have a sketch that use the eeprom. It checks to see if the values have been set first and if not, defaults to NETWORKID = 1 and a NODEID = "random(2,255)". Not perfect but should make setting up new node really simple, even if you do it via serial. I am also saving the Frequency in the eeprom so that I can use the same sketch on moteinos with different radios.

(NOTE: looks like I will still have to set the initial frequency in the eeprom before using this sketch as I don't want to risk messing up a radio yet and force to to a frequency to was not designed for to do initial configuration. I.E set a 433 radio to 915. although I will try it at one point just to see if it works :-) )

Then I added some commands to allow the ID to be set via Serial and Radio. So far it is working but i have to clean up the code a lot before I post it. I also want to work on the Gateway some more as i think i have a way for it to self assign node IDs. so you would only need to set your gateware in "setup mode" and tell it want network ID you want to use, then turn on your nodes and then would "self register" (or DHCP as you put it). then when your done setting up you put it in run mode and the gateway resets and using the Network ID you told it to.

After that will be to add in the wireless programing code and ability to log sensor data to the SPIFlash for debugging or disconnected/intermittent connectivity.

[KanyonKris]

Jarrods, that's good stuff. I look forward to seeing your code.

john k2ox has done some similar work you may want to look at - http://lowpowerlab.com/forum/index.php/topic,277.0.html

[PhilSantos]

Thanks... It is funny you mentioned that cause I just finished testing that EEPROM library and it works just fine. So I am taking that route. I like the low power idea but not for the power saving, but because it frees up the serial. when your programing a node you cannot receive data.

I am looking at using and Inductive charger built into the case to recharge a 850mah LiPo and a small solar panels for the outdoor notes (will be a while before i build those however).

Thanks for the advice!

I really liked your idea of using inductive charger.. I am working on similar sort of project so it might work well for me.