RFM69 10bit node addresses

I made some changes to the RFM69 library to support 10bit node addresses, and you should see version 1.3.0 available in your Arduino IDE Library Manager. Previously the standard was a 1byte (8bits) address for sender and receiver, a total of 2 bytes used in each packet for addressing. I wanted to avoid using another 2 bytes to increase addressing to 16bit (65k addresses) because having that many nodes is impractical, and 2 more bytes is an “expense” most users won’t need. So I used the bottom 4 bits of the CTL byte, to add 2 bits to each of the existing to/from address bytes. The network addresses are kept at 1byte (8bit).

This yields an address space of 1024 total addresses. Address 0 is now reserved for the BROADCAST address, and you are left with 1023 possible addresses. A gateway is suggested to be kept at address 1 and the rest of the nodes should occupy incremental addresses.

There should be no impact on anyone not using more than 255 nodes as before. One required change for those wanting to use addresses larger than 1 byte (255) is they need to use uint16_t variables in their sketch. I’m looking forward for constructive feedback and any bug reports at this time. The ListenMode part of the code (which is experimental) was left untouched and it only supports 8bit addressing as before.

Here is the latest RFM69 packet header structure including a breakdown of the CTL byte:

Other important updates:

  • The BROADCAST address was changed to 0 (from 255 previously).
  • The previous RFM69 v1.2.0 included SPI Transactions, another important change to the library.
  • After it was brought up in the forum that Moteino AVR sketches compile to larger sketch size than Arduino, I recently released a new AVR Moteino Boards definition (based on Arduino 1.6.23 AVR boards) which results in significant compiled size reductions for all Moteino sketches, please give this a try. In Arduino IDE’s Boards Manager you should be able to easily upgrade to the latest version.

RFM69 and SPIFlash libraries released via Arduino LibraryManager

The LowPowerLab RFM69 and SPIFlash libraries are now published to the Arduino Library directory, and available to install in the IDE’s Library Manager (under Tools>Manage Libraries). Note that if you already have these library installed manually, you may need to remove it and re-install using the manager:

As new versions are released, you can update to the latest or switch back to an older version if you have a need to do so. If you’re new to using the Library Manager, it’s worth to read the official Arduino Library guide, and also check the Library Manager FAQ, to understand how Arduino libraries work, and how manage them (adding, switching, updating, manual library installation, deleting local libraries, etc).

As usual, please report any issues or bugs in the forum or open a Github issue where appropriate.

Custom ArduinoIDE: Boards & Ports toolbar add-on

Have you ever done testing of lots of different boards with the Arduino IDE?
Were you frustrated with having to search your target board and port in the Tools>Boards submenu every time? Do this a few dozen times and you realize this is a major pain and time waste. Especially true if:

  • your boards menu is getting ever longer with every version and new package
  • with every new board there is a new serial port to select
  • you are in a hurry, make a wrong selection, get compile/upload errors, start selecting your board/port from scratch, face-palm!

I think if I add all the time I spent doing that in the past decade I could go on a short vacation in the carribean, wouldn’t that be nice!

I spent some time dusting off my Java skills and understanding the Arduino IDE source code and made this little add-on toolbar:

Here’s a video overview of how this works:

Continue reading

Gateway app updates

There are some notable changes and some new features for the Gateway app – these are published in this Github release. Below is a video overview and a summary of changes:

New settings:

  • serial baud is now 19200 (old: 115200), this is to eliminate any serial corruption when lots of serial data is going in/out to Moteino/MightyHat
  • keepFilesOnDelete (default: false) ensures the binary log files are removed when node/metrics are removed
  • graphMaxPoints – graph data point resolution (total points graphed from raw data)

image

New features

  • List reordering (desktop only) – drag & drop nodes to reorder list & broadcast to all clients:
    image
  • Graph legend div reformat + count of total raw data points that produced a graph are now displayed on top left.
  • Also there is a new Export-Raw-Data button that generates a CSV file from the selected time window. This was requested here.
    image

Wireless Programming just got 50% faster!

The Wireless Programming GUI app that I introduced in an earlier post is now at version 1.2 (get exe here). It features the ability to bundle up to 3 HEX file lines per RF packet (instead of the default 1), which yields a significant transfer speed increase. Let’s just do the TL;DR first and gaze at the results of transferring a compiled sketch of 13,858 bytes:

  • running with 1 line per packet (the same as before) yields 866 packets and ~43s OTA time:
  • running with 3 lines per packet yields 290 packets and ~18s OTA time:

That’s an average 58% speed gain, not too bad!
By comparison, the same sketch uploaded directly via FTDI/serial takes about 13s, so much closer to that figure than before. I think a few other techniques could be used to squeeze yet more speed (increasing RF bitrate!) but that’s for another day.

Note that the same exact data is transferred, except in fewer longer packets. I had to make some changes to the WirelessHEX69 library to support these longer packets. Also I fixed some bugs that emerged during this development, see the github commit for all the changes.

To support the new GUI v1.2 speed increase you only need to update your WP programmer Moteino with a new version of the WirelessProgramming_gateway sketch (to refresh the WirelessHEX69 code) to make it support the faster WP speed (ie also update the WirelessHEX69 lib!). Everything else is completely backwards compatible, and you can use the new GUI v1.2 with an older WP programmer Moteino if need be (1 line per packet only), and also all old target nodes that are OTA/WP programmable are not affected at all and can be updated at the new speed. FWIW you can actually use 1 or 2 (not sure why you would) or 3 lines per packet (the new default going forward). Experiment and see what your own speed gains are! Please report any bugs or issues in this forum!

RFM69_ATC: Automatic Transmission Control

There is now a new extension to the RFM69 library. It’s called RFM69_ATC aka Automatic Transmission Control. Many thanks to Tom Studwell who implemented this and shared it in the forum. The basic idea behind this extension is to allow your nodes to dial down transmission power based on the received signal strength indicator (RSSI). For instance a sleepy battery node like a MotionMote sits still inside the home and usually has a very strong received signal, somewhere in the range of -30 to -40dBm when transmitting at full power. You could manually tune that down using radio.setPowerLevel(..) in code but its tedious and is a static change, if you move the node or conditions change it will not be smart to adjust the power level to the new environment. However with RFM69_ATC this is done automatically for you, you just need to indicate a target RSSI. On each packet sent and ACK received (using sendWithRetry(…) is required), the node analyzes the actual RSSI and continuously adjusts its own transmission power level of the RFM69 transceiver to attempt to match the target RSSI (+ or -). This way that end node is only “loud” enough to be heard by the gateway, not much louder. Close by nodes can “whisper” while nodes farther away or with more obstacles will “speak up” as necessary but avoid that default fixed maximum “screaming” level. This is “polite” in terms of “RF pollution” and efficient in terms of power consumption. Even non-battery nodes where power is no problem should implement this for the sake of the “polite” factor.

I have updated the Node and Gateway examples to have ATC enabled, and also the MotionMote example is now ATC enabled, in this case with a target RSSI of -90dBm. The noise floor is somewhere around -100dBm with the default RFM69 lib settings, so for static nodes that won’t experience a lot of movement or temperature drifts, a -90dBm target is pretty safe. That will keep the transmitter power to a minimum and save power. We know that the greatest toll on a battery powered node are the spikes of current used when the transmitter is active (up to 130mA for RFM69HW). Reducing the transmit power level to the minimum required will exponentially reduce that spike and result in longer battery life and a more “quiet” sensor network that won’t reach across your whole neighborhood. This is really awesome!

Here is a sample transmission using the Gateway and Node examples linked above. Note how the node starts transmitting at full power, then dials down power to match a test RSSI target of -68dBm. This is output from the Gateway end:

ATC

There are a few required changes to a sketch where you want to use ATC:

  • you must #include <RFM69_ATC.h> in addition to #include <RFM69.h>
  • you must use RFM69_ATC radio; instead of RFM69 radio;
  • for the gateway/receiver end the above two changes are sufficient, for the end node that does the power level adjustment you must also do the following:
  • in your setup() function (after all RFM69 initialization is complete) call this function to set your target RSSI: radio.enableAutoPower(targetRSSI);
    • targetRSSI is a negative integer, should be from very strong (-30) to very weak (-95). Usually you would want to be closer to the noise floor end (-100) since you want to reduce transmit power to the bare minimum
    • this is a one time call that enables the dynamic adjustment of the output power on that node
  • you must use radio.sendWithRetry() instead of just radio.send() or a combination of radio.send() and radio.receiveDone() for the radio to be able to receive the important headers from the gateway that tells it how to handle its power, this was done in the MotionMote example linked above

Once these changes are implemented, the node will start to progressively dial down power (assuming it starts transmitting at full power) with each packet sent, until the RSSI meets the target. When the RSSI is below the target power is dialed up again and so on, in an attempt to stay as close to the target as possible.

In the examples that I mentioned there is a pattern that I followed by implementing a define directive (#define ENABLE_ATC) which when left uncommented will enable ATC at compile time, if removed/commented the sketch will run normally without ATC. As always, bug reports, suggestions and contributions are welcome, the forum is the best place for that purpose.

I must mention here that at this time I have only partially implemented his variant with some adjustments to Tom’s implementation (I left the differences commented out in my version). For now I chose to leave my power control resolution as is. He actually went into more detail allowing a finer control of the power control – this is because of the differences between RFM69W and HW which handle output power control differently. If you’d like to try his variant with that extra control check this forum post or his Github repo. Thanks Tom for your great contributions and inspiring this piece of work and also the awesome forum projects you’ve posted!

Gateway Source Code Released

A few weeks went by since I blogged about the new Gateway interface. I managed to get the code ready and I posted the links on the dedicated gateway page where I prepared a guide to install the stack needed to support it (this will be improved). The code itself and sources are published at Github. There you will find the old interface sample I had posted a long time ago (in the OLD subdirectory, kept there for reference); the files are replaced with the new interface files.

There are 2 directories where you will need to copy files from here after you’ve completed the setup of your Pi from this guide (I may release a Pi image that has everything setup and ready to go). Please note: when you download stuff from Github you must use the Download Zip button in the repository rather than downloading individual files.
First there is the /var/www /default directory, you should copy these files there:

Second there is the /home/pi/moteino directory, you should copy these files there:

Once the gateway starts running it will create the databases for you and start logging data as it comes in from your remote nodes through the gateway Moteino attached to your Pi’s serial port configured in the gateway.js script.

This post is not meant to replace the guide found at the lowpowerlab.com/gateway page. I will post further details there. I created a new forum for discussing this and other gateway related topics so please post your questions there.

Moteino now on CodeBender.cc!


IMPORTANT NOTE: unfortunately codebender announced they are shutting down on or after Oct. 20, 2016. Hence forth this page is only kept for reference and it will eventually be removed. The codebender programming instances below might become inactive, display an error or show a blank space once codebender shuts down these services. You are encouraged to start programming or move your code in the Arduino IDE.


This is really awesome news!
The guys at codebender.cc have done a great job creating an online place to create, edit, store, share your Arduino sketches. It’s very easy to install and use right in your browser!

UPDATE: Check the new Moteino Programming Guide.

And now Moteino has been added to their fleet of supported devices, along with the RFM69, SPIFlash libraries. Once your Moteino is connected to your computer, you can edit your sketch, or load the example(s) of the many integrated libraries (like the Node example in the RFM69 library) edit the settings to match your hardware and upload it to your board. Another awesome feature of codebender is embeddable sketches, that’s right, any sketch can be embedded in another web page, like the Node example below, you can upload it right from this blog post (see below for browser plugin requirements, or check this getting started guide):

To be able to use CodeBender you will need to create an account, then install a browser plugin that allows access to your computer’s serial ports, which in turn make it possible to reset and upload sketches to your Moteinos (or whatever other board you may have and they support). Also you need to choose your target board, protocol (USBtinyISP for Moteino) and serial port it’s attached to, then click “Run on Arduino” and that will compile/upload the sketch. That’s about it. Please check their getting started guide as well.

I warmly recommend using CodeBender with Moteinos since it’s easy and much nicer than the clunky Arduino IDE, and it’s probably going to get even better as they add more features. Plus I will work with them to ensure the libraries are in sync with the latest releases, and new examples will also be posted on CodeBender. Of course, you can always add your own sketches to your CodeBender repository, just like you did in Arduino, or you can clone an example and load your own settings and modify the code as you like. I will probably start using embedded sketches whenever I want to share code since it allows CodeBender users to upload it right from this blog rather than going to GitHub, copy pasting, making sure they got all the latest updated libraries, etc.

To celebrate this great milestone I am reducing the pricing on all Moteinos by $2, a saving of 10-15% depending on your options, to last through the end of December!

Happy CodeBending!

ATXRaspi code update

Hello ATXRaspi users!
The previous version of the script was running OK, but there was an obscure side effect which I had overlooked. The script was invoking “sudo sleep” in between checks of the shutdown signal from ATXRaspi, and this in turn was generating new entries in /etc/log/auth.log every time, yikes! Thanks to Chris L. for pointing this out. The problem with that is the Pi’s that run from SD cards will have continuous activity to the file system and onto the disk and hence increase the risk of data corruption after very many disk accesses to the flash media. As Chris noticed, this also produces over 300MB of log per week, which gets archive into about 7MB (gzipped), not terrible in terms of disk space usage but still highly undesirable. I have changed the script to remove this side effect and reflected the install changes on the ATXRaspi page.

UPDATE: A big improvement is also the change to a single setup script that needs to be run to install the shutdown script. No more dependency on WiringPi. Just type 2 commands and reboot your Pi and you’re done!

Another notable change was that I moved the install location from /home/pi directory to the /etc directory where system code should generally reside.

I encourage all existing ATXRaspi users to upgrade to the new script. Since this is the same script that would be used by MightyBoost+bare Moteino that emulates ATXRaspi behavior, they should use the same new script. As always, please feel free to report any issues and improvements, the forum is a good place to do so.

Library updates & MoteinoMEGA support

Following the MoteinoMEGA release, the RFM69 library and examples have been updated to support MoteinoMEGA, there are significant changes to the library and you should get latest. The MoteinoMEGA Arduino Core has been released and needs to be installed in the Arduino/hardware folder (stable v1.0.5 tested working) before uploading to MoteinoMega. The Eagle source and schematic/board files are published in the LowPowerLab github repository. The Moteino page was updated to reflect the addition of the MEGA to the Moteino family.

The Wireless Programming library and examples are updated to fix bugs and support the MEGA as well. Another important change is the introduction of radio.ACKRequested() function in the RFM69 lib which should be used instead of the radio.ACK_REQUESTED variable. This ensures that ACKs requested by a broadcaster are ignored and only targeted ACKs are responded to. Please update your examples for proper wireless programming behavior. All examples have been updated to reflect this change. Also the SwitchMote sketch has been updated.

I consider this a major release and hence your constructive feedback and notification about anything that was omitted is appreciated.