RFM69_ATC: Automatic Transmission Control

There is now a new extension to the RFM69 library. It’s called RFM69_ATC aka Automatic Transmission Control. Many thanks to Tom Studwell who implemented this and shared it in the forum. The basic idea behind this extension is to allow your nodes to dial down transmission power based on the received signal strength indicator (RSSI). For instance a sleepy battery node like a MotionMote sits still inside the home and usually has a very strong received signal, somewhere in the range of -30 to -40dBm when transmitting at full power. You could manually tune that down using radio.setPowerLevel(..) in code but its tedious and is a static change, if you move the node or conditions change it will not be smart to adjust the power level to the new environment. However with RFM69_ATC this is done automatically for you, you just need to indicate a target RSSI. On each packet sent and ACK received (using sendWithRetry(…) is required), the node analyzes the actual RSSI and continuously adjusts its own transmission power level of the RFM69 transceiver to attempt to match the target RSSI (+ or -). This way that end node is only “loud” enough to be heard by the gateway, not much louder. Close by nodes can “whisper” while nodes farther away or with more obstacles will “speak up” as necessary but avoid that default fixed maximum “screaming” level. This is “polite” in terms of “RF pollution” and efficient in terms of power consumption. Even non-battery nodes where power is no problem should implement this for the sake of the “polite” factor.

I have updated the Node and Gateway examples to have ATC enabled, and also the MotionMote example is now ATC enabled, in this case with a target RSSI of -90dBm. The noise floor is somewhere around -100dBm with the default RFM69 lib settings, so for static nodes that won’t experience a lot of movement or temperature drifts, a -90dBm target is pretty safe. That will keep the transmitter power to a minimum and save power. We know that the greatest toll on a battery powered node are the spikes of current used when the transmitter is active (up to 130mA for RFM69HW). Reducing the transmit power level to the minimum required will exponentially reduce that spike and result in longer battery life and a more “quiet” sensor network that won’t reach across your whole neighborhood. This is really awesome!

Here is a sample transmission using the Gateway and Node examples linked above. Note how the node starts transmitting at full power, then dials down power to match a test RSSI target of -68dBm. This is output from the Gateway end:

ATC

There are a few required changes to a sketch where you want to use ATC:

  • you must #include <RFM69_ATC.h> in addition to #include <RFM69.h>
  • you must use RFM69_ATC radio; instead of RFM69 radio;
  • for the gateway/receiver end the above two changes are sufficient, for the end node that does the power level adjustment you must also do the following:
  • in your setup() function (after all RFM69 initialization is complete) call this function to set your target RSSI: radio.enableAutoPower(targetRSSI);
    • targetRSSI is a negative integer, should be from very strong (-30) to very weak (-95). Usually you would want to be closer to the noise floor end (-100) since you want to reduce transmit power to the bare minimum
    • this is a one time call that enables the dynamic adjustment of the output power on that node
  • you must use radio.sendWithRetry() instead of just radio.send() or a combination of radio.send() and radio.receiveDone() for the radio to be able to receive the important headers from the gateway that tells it how to handle its power, this was done in the MotionMote example linked above

Once these changes are implemented, the node will start to progressively dial down power (assuming it starts transmitting at full power) with each packet sent, until the RSSI meets the target. When the RSSI is below the target power is dialed up again and so on, in an attempt to stay as close to the target as possible.

In the examples that I mentioned there is a pattern that I followed by implementing a define directive (#define ENABLE_ATC) which when left uncommented will enable ATC at compile time, if removed/commented the sketch will run normally without ATC. As always, bug reports, suggestions and contributions are welcome, the forum is the best place for that purpose.

I must mention here that at this time I have only partially implemented his variant with some adjustments to Tom’s implementation (I left the differences commented out in my version). For now I chose to leave my power control resolution as is. He actually went into more detail allowing a finer control of the power control – this is because of the differences between RFM69W and HW which handle output power control differently. If you’d like to try his variant with that extra control check this forum post or his Github repo. Thanks Tom for your great contributions and inspiring this piece of work and also the awesome forum projects you’ve posted!

New products!

I am quite happy to announce a few new products (or new revisions), some of which people have been asking for a long time, now finally available at the shop!
First there is the IOShield which is now available as an assembled PCB with optional “24VAC input package” which allows you to place it in places where you have 24VAC input, like I did in the wireless sprinkler controller project.

Then there is a small BellMote run with OSHPark purple PCBs. Completely automate your doorbell with this kit which allows you to detect, trigger, and disable your doorbell, all wirelessly via Moteino gateway.

Next up is the SonarMote which finally has a new revision (R2) which addresses some bugs, has a simpler BOM and is even lower power – just 10uA average when the Moteino is properly slept – sample sketches here. A wide range of applications are possible, from parking sensor, inventory control, sump pump monitor, and others.

Finally we have the MiniBoost which is a small SIP form boost regulator that yields 5V from an input voltage of 2-4V. Up to about 1A is possible from a charged LiPo battery (4V) and will easily carry a moderate load like a RaspberryPi. It uses the same boost regulator as the MightyBoost but the SIP size makes it very versatile for breadboard projects. It’s perfect for projects where you have a main working voltage of 3.3V but you have some sensor that needs a hefty 5V source. Comes with onboard green power LED. It has is the same pinout and size as the legendary 7805 regulators:

It takes a long time to go through prototype stages and then manufacture electronics of high quality, all as a small operation like I have. I hope to get more time to put more documentation and assembly instructions together for these kits and boards. Some of them have already been introduced as prototypes and have a fair amount of information available. There are also other things I’d like to blog about, just not enough time, stay tuned for more!

Gateway software update (v6)

The Moteino Framework Gateway software stack was updated and new sources are available at their Github repo, and also a new image with the changes is released (for the image link see the Gateway image page). Let’s call this v6, since it’s the 6th gateway image I’m releasing. This release addresses some bugs and adds some new features:

  • Added ability to make HTTP requests from a node using the node request module. This means you could include controls that trigger a request to some IP in the LAN/WAN and does something. This is really cool because it means we can have non-Moteino based nodes that speak HTTP instead of RF, like commercial wifi thermostats. Which leads to …drum roll
  • …the addition of a specific type of open API wifi thermostat (Radio Thermostat CT50) that allows full control of HVAC equipment. I will review it and blog about this addition whenever I get some time. Also added a new node type and icon for the water meter:
    Gateway_ThermostatWaterMeterIcons
  • adding (injecting) a non-moteino node (ex for the thermostat I mentioned above) to the gateway nodes collection can now be done as easily as this (in the terminal screen, type a new node ID and “NEW” in the msg box, click SEND):
    Gateway_NodeInject
  • Graph auto scaling, requested and discussed in this forum topic. The yaxis scale will automatically scale depending on the data viewed, very useful for highly variable or granular metrics, see the thread for screenshots of before/after and specific code changes to allow this.
  • fixed a bug in the new log storage engine where negative data was stored as unsigned 32bit integers, thus being extracted as positive. Negative data should now be welcome.
  • don’t allow graph live incoming data values when the graph was panned/zommed into a specific region
  • migrated from sliders to top bar toggle buttons for node-visibility, metric-pinning, enable-graph. Sliders were a pain to work with.
    Gateway_pinnedGraphed
  • lots of other refactoring and optimizations especially in the metrics.js and UI areas

I hope you enjoy the new release.

Gateway log engine upgrade

So far the Moteino Framework Gateway server application has been using neDB for storing node metadata/settings, and also to store historical metric data logs. Since neDB keeps all data in memory, it gets slower over time as data grows. Metrics with frequent updates could take 10sec to load 1 week worth of data (a few thousand data points). Visually it’s completely pointless to load 10K points on a graph that’s 1000px wide (you can’t see 10 points in 1 screen pixel). So a lot of wasted time and performance to search/move/display all that “invisible” data.

I researched a few options to improve storage/graph loading time. The list is below and if you want more details about each and about this effort see this page.

  • I tried implementing a binary search for neDB to get the graph data, but it wasn’t much better, the core engine of neDB is too slow for this purpose
  • I searched online other databases to store timestamped sensor data. There are expensive enterprise engines and also open source toolchains like Apache Hadoop – this looks very nice but it’s complex and a steep learning curve (Pi support?).
  • There’s mongoDB which is wonderfully capable and similar to neDB (or neDB similar to mongo rather) but still not officially supported on the Pi so it’s a no-go for now.
  • There’s Timestore – a C++ app by Mike Stirling. This is a great standalone engine but I have a requirement: I cannot assume when a data point comes in to be stored, it can be highly variable and I can have a thousand points in 1 hour, then nothing for a week. So I need to store the timestamp.
  • The guys over at OpenEnergyMonitor have done a lot of work based on Timestore. They had the same problems to solve as I did. Their EmonCMS app used MySQL to log sensor data, which was increasingly slow as tables got very deep. So they created PHP storage variations based off Timestore to allow logging variable interval data. Thanks guys for sharing your work and research results!

After considering all these options, I decided to build my own node.js storage engine based on bits and pieces from Timestore and the variable interval engine from OEM. The main reason is that it needed to be all in node.js. Doing it in PHP and then hitting a PHP script from a node.js socket app to query/post data felt very wrong (not sure how performant either).

The result is a much faster binary storage engine that can query and search and aggregate a month of data in about 500ms (on a Pi B Rev1 256RAM!). That’s more than a magnitude faster than neDB. However neDB will still be used to store node metadata, it’s perfect for this purpose and will be very fast and use very little memory.

The Gateway guide has been refactored to include this new engine details along with a few other changes and features (mostly UI):NodeView2

  • vastly improve graph loading times regardless of zoom and span viewed
  • add nice customizable tooltips to data points (customize in metrics.js)
  • add customizable legends on the graps (customize in metrics.js)
  • using upstart for starting/stopping/restarting the gateway.js script. This replaces the init.d way to start it and also ensures the gateway script will be respawned in case it crashes
  • add Font-awesome icon pack (over 580 awesome icons to use with jQuery-mobile) and use some of these new icons for some buttons
  • toggling node/metric visibility and metric graphing is now done via buttons instead of sliders which were not working very well on mobile devices. This also removes clutter on the node and metric pages

Upgrading from old neDB gatewayLog.db data

If you’ve used my Gateway interface before, you probably have log data stored as JSON in gatewayLog.js. No problem. Just save your existing gatewayLog.db, upgrade your Pi to the latest image, copy the gatewayLog.db into /home/pi/moteino, then run this upgrade script in the same directory – it parses your gatewayLog.db data and puts it into binary files in a db subdirectory, ready for the new gateway.js script to use. Otherwise gateway.js will just create logs from scratch as data comes in.

UpgradeGateway

It was a lot of work to get this right, I’ve been tweaking all this for the past few weeks. It’s free to use non-commercially, I hope you like it, I think it’s a major improvement overall, both back-end server and front-end UI. If I missed something in the guide or there’s any bug let me know. There’s more good stuff coming, just not enough time to document and blog it all. Cheers!

DoorBell Mote regulator update

The last BellMote prototype ended up having 2 LDO linear regulators, 12V and 3.3V feeding from the 12v one. I was split on using LDOs going forward. The upside is they are cheap, even 2 of them are under $2. The downside is they are inefficient and run warm/hot depending on the load. I could have gone with a single 5V LM7805 which will run even warmer, the higher the gap between Vin and Vout.

Digikey-ing around, I found these two switching regulators that are drop in SIP replacements for any LM78xx series LDOs: a RECOM 0.5A output with 7-28v input range ($2.8), and a Murata 1.5A 7-36v input range ($4.25). I tried both, and below are the results:

I won’t repeat the captions here but the clear winner is the Murata (plenty stocks at Digikey) it’s quite a bargain given the advantages. I tested it on 24VAC as well and it does not heat up at all either so it will be perfect for some coming interesting projects I have in the works that will feed on 24VAC. The RECOM regulator is just $2.8, still good enough for 16VAC but I prefer the Murata for further 16-24VAC powered projects. That said, I will offer these for sale (2 of them, one RECOM, one Murata, and I will replace the LDOs with a Murata and keep the third one). I may make a kit of this, not sure yet. For now you can grab these two from the first prototype batch, they will be worth millions when LowPowerLab is like Apple and me like Steve Wozniak 30 years from now (-:

Gateway Source Code Released

A few weeks went by since I blogged about the new Gateway interface. I managed to get the code ready and I posted the links on the dedicated gateway page where I prepared a guide to install the stack needed to support it (this will be improved). The code itself and sources are published at Github. There you will find the old interface sample I had posted a long time ago (in the OLD subdirectory, kept there for reference); the files are replaced with the new interface files.

There are 2 directories where you will need to copy files from here after you’ve completed the setup of your Pi from this guide (I may release a Pi image that has everything setup and ready to go). Please note: when you download stuff from Github you must use the Download Zip button in the repository rather than downloading individual files.
First there is the /var/www /default directory, you should copy these files there:

Second there is the /home/pi/moteino directory, you should copy these files there:

Once the gateway starts running it will create the databases for you and start logging data as it comes in from your remote nodes through the gateway Moteino attached to your Pi’s serial port configured in the gateway.js script.

This post is not meant to replace the guide found at the lowpowerlab.com/gateway page. I will post further details there. I created a new forum for discussing this and other gateway related topics so please post your questions there.

Gateway Dashboard Overview

I’ve been talking about a new gateway interface for a while in the forum. I will release more details and the source code soon. Here’s an overview that I’ve put together the past week,  and some screenshots below with graphs and other available features:

Here are some more screenshots:

MAILBOX_DSC_7214DOORBELL_DSC_7220
MOTION_DETAILSMOTION
SWITCHMOTE_DETAILSWITCHMOTE_GRAPH
GARAGE_DETAILGARAGE_GRAPH
DOORBELL_DETAILSDOORBELL_GRAPH
WATER_GRAPHSUMP_Graph2

This content will be updated and improved on the dedicated Moteino Gateway page where all the source code and details will be published.

New ATXRaspi reboot function

ATXRaspi will start to ship with a new reboot function in addition to the shutdown function it had since inception. This was implemented because it was a cool feature to have and also suggested by several ATXRaspi users. See video above for a full overview and setup guide for ATXRaspi.

This will be in revision R2.6 boards but until that is released, revision 2.5 boards that have the reboot function will have a blue dot on the main chip (see photo above). The differences are the following:

  • to reboot: hold the button pressed for at least 0.5s and and less than 2s. The button backlight will dim once the reset threshold is met. Release the button and ATXRaspi will emit a 500ms HIGH pulse on the SHUTDOWN signal pin. It will then blink the button backlight for up to 1 minute while waiting for the Pi to reboot and the BOOTOK signal to be restored by the shutdowncheck script (to become HIGH again).
  • to shutdown – nothing changes: hold the button at least 2s. As before, the button backlight pulses slowly while the Pi shuts down. Once the shutdown is complete and BOOTOK signal goes LOW, ATXRaspi waits a few more seconds and cuts power off to the Pi.

The shutdownchecksetup.sh install script was updated to support the reboot function and is backward compatible with all previous ATXRaspi revisions. Here’s a photo of the required wiring to the GPIO and an example wiring of the chrome button offered at the LowPowerLab shop:

Limited SonarMote and PiGateway offering

I worked on a project for inventory control and the hardware left over from that project is now available for sale for those interested to save time on assembly. There are a handful of assembled SonarMotes and a PiGatewayThese limited LowPowerLab artisan electronic creations are available in the webshop (all SOLD).

SonarMote is a project I worked on for some time last year for private projects, but was never released to the public mostly because of the economics of manufacturing it and the end cost of the whole kit. But otherwise they are great for distance measurement, sump pump or liquid level monitoring, parking sensors, and general purpose distance sensors, battery operated and wireless via onboard Moteinos, and easy drop-ins into your Moteino framework. For instance, I am using one of these to monitor my sump pump. Now the rest of the few assembled units can be all yours. They run on LiPo batteries (1500mAh included in case) and are rechargeable and programmable via USB port just like the MotionMote. Note all are RFM69 868-915mhz. The rest of the specs are posted on the product page.

The PiGateway is a unique build that includes an ATXRaspi and LED Switch, Moteino with RFM69HW 868-915mhz, 4GB SDcard with Raspbian and Nginx-node webstack, 2.1mm jack power adapter and slick black translucent acrylic case.

New SwitchMotes available

I’ve released some new SwitchMote kits after some requests by different users of SwitchMote. All SwitchMotes are wireless AC actuators, some designed to replace conventional light switches for the purpose of automating household light switching. Specifically there is a new dual 10A relay SwitchMote (and PSU):

There is a new assembly guide for this specific variant posted here. Here are some photos if it assembled and compared to the original SwitchMote:

Also there is a new single 30A relay SwitchMote PSU for heavy AC loads. The PCB for this particular one has double copper thickness to support the loads (2oz copper). Note that the relay has parallel posts at the top for heavy duty connectors as an option.

The demonstration of this PSU has already been posted in a video a few days ago:

The demo in the video uses the TxRxBlinky sketch. All of these are available at the Low Power Lab webStore. The SwitchMote guide page has been updated to include these new variants and the SwitchMote sketch was updated to support the new dual relay SwitchMote 2x10A.