Gateway V9 Released

The Pi Gateway software is now at v9.0.0, this release is a major new feature and bug fix release. This blog entry serves as a change log and feature review. Below are the main highlights.

New Node button & sample metric generators (ex: Internet Speed polling event)

Clicking New Node you may add a numeric ID node. This could be a future RF or LAN node, or a dummy node to hold data such as your internet speed over time (add the Internet Speed event and enable it).

For RF nodes, any node that starts sending data through the serial port attached RF gateway, will automatically generate a new node. For RF nodes, node IDs up to 1023 are now possible thanks to 10bit addressing. Below you can also see a node with ID 999 that reports temperature. 

MultiGraphs

A new Multigraph button on the node page allows generating a graph containing multiple metrics right on the node page. This enables you to easily compare the various metrics of a node chronologically. Clicking the far right remove button under the graph allows you to generate another graph with a different combination of metrics. There is a practical limit to how many metrics you can show since each metric can contain a lot of data and generates a separate vertical legend, so be reasonable and avoid pushing limits. Here’s such a graph showing the various metrics of an RF node:

Desktop graph scroll-zoom & double-click zoom

You may now scroll-zoom with your mouse wheel over a graph (works in all graphs) and it will zoom in and out. You can also double-click and it will zoom in. On mobile devices double tapping will zoom-in.

Edit and Delete metric data

Graphs now show lines between data points instead of bars by default. Still, sometimes it can be confusing looking at lines or bars, where the actual data points are. So you can now turn on data points. Additionally, you can edit and remove any single data point (click on specific data point to edit/remove) or range of data points (select range of data points to remove or change to a new value) if you wish by enabling the edit/delete mode using two buttons below the metric graph. Data points are automatically enabled when editing data.

Easy node image change

You can now change a node’s icon very easily to any existing icon (under the /www/images folder), or upload a new image of your choice (uploaded to /www/images/uploads). Icons should be 120x120px, if larger they are resized. Clicking the node icon:

This brings up the node icon change dialog (pick existing or upload new):

Other icon changes:

  • some default icons have changed (ex. CT-50 thermostat)
  • for consistency, all default icon nodes are now prefixed with icon_
  • rssi icons renamed to rssi_n with n=[1, 7]

New HTTPEndpoint for posting data from LAN

You may now submit data to the Gateway from any LAN device, thus enabling the use of WiFi (ESP anyone?) and network connected nodes, and also essentially removing the need of using only RF nodes. For instance:

https://raspberrypi/httpendpoint/?id=1234&MOTION&F=77.77

will generate a new node (if non existent) with ID=1234, and generate two new metrics (MOTION, and temperature). If an ID is not provided, then the IP of that node becomes the ID. The generated node will always have a new _ip property which contains the IP of the sender. Other examples:

  • /httpendpoint/?MOTION&F=12    ->  ID=(valid IP)
  • /httpendpoint/    -> no update, just lastupdated timestamp

The response will be JSON of this form:
response: {"status":"success","message":"SUCCESS!","matchedMetrics":2}

Run in internet isolated LAN

All script files are now loaded locally and there is no longer a dependence on an internet connected RaspberryPi to fetch these scripts. So there is no concern running this completely disconnected or behind firewalls etc.

Title setting & Menu changes

You can set your own custom title from the settings page, don’t forget to click Save to apply changes. The menu now shows the Gateway app version, when the database was last compacted (done once daily automatically), and an option to compact the database on demand.

You can now quickly restart the app via Ctrl+Atl+Shift+R, or from the Menu as before.

Run it as a Mobile app

You can now run the application as a mobile app. A manifest.json definition that loads automatically with the web page allows you to create a shortcut via the Add To Home Screen option in your mobile device’s browser. Then clicking that icon on your home screen opens the web page like a mobile application without the extra browser bars.

New default metrics

  • It’s sometimes desirable to compare the RSSI of an RF node with the transmit level of an RF node. The new X:n metric with n=[0, 31] allows this to work nicely with the RFM69_ATC library extension and you can see the transmit level drop or rise based on the RSSI change.
  • A new TYPE:nodeType which matches the exports.motes definition of a mote in metrics.js allows to automatically pick that mote’s icon and other features like buttons. Example: when a SwitchMote is powered up, it can send a 1 time TYPE:SwitchMote token to indicate it’s a SwitchMote. You can always later override and pick the node type from the node type dropdown

Other misc changes

  • gateway.service is now run as pi:pi, and systemd will keep retrying to restart it if it crashes
  • fixed genNodeIfNoMatch setting
  • the /www/images/uploads directory is now chowned by www-data:pi during setup
  • node name/title added on the metric page – this helps determine which node’s temperature metric you may be looking at (vs. just “Temperature”)
  • ordered Node/Event type dropDowns (A-Z)
  • reworded and simplified licensing terms

Installation and upgrade path

This being such a major change with many new/renamed/deleted files, it’s a good idea to reinstall the application from scratch on a new raspbian image rather than try to do an in-place manual upgrade. This also ensures you will be running the latest and best raspbian, nginx, PHP7, nodeJs & packages, etc.

To transition your data to V9, backup the content of the /home/pi/gateway/data/dbfolder, then make a complete image backup of your currently functional previous version and store that until you are sure V9 runs smoothly on your Pi and all your old data is picked up.

Then follow the Gateway software setup steps in the official guide to install V9 to the latest raspbian image.

Once the V9 is running, to load your old data, you will need to sudo systemctl stop gateway, copy/overwrite the saved data into the new /home/pi/gateway/data/db folder, and then sudo systemctl restart gateway, and you should see all your old nodes and their data.

For any issues and bugs, please use this forum or you may submit a PR to the Github repository. Note that this change log may be edited as any loose ends are tied and before an official release is created.

RFM69 10bit node addresses

I made some changes to the RFM69 library to support 10bit node addresses, and you should see version 1.3.0 available in your Arduino IDE Library Manager. Previously the standard was a 1byte (8bits) address for sender and receiver, a total of 2 bytes used in each packet for addressing. I wanted to avoid using another 2 bytes to increase addressing to 16bit (65k addresses) because having that many nodes is impractical, and 2 more bytes is an “expense” most users won’t need. So I used the bottom 4 bits of the CTL byte, to add 2 bits to each of the existing to/from address bytes. The network addresses are kept at 1byte (8bit).

This yields an address space of 1024 total addresses. Address 0 is now reserved for the BROADCAST address, and you are left with 1023 possible addresses. A gateway is suggested to be kept at address 1 and the rest of the nodes should occupy incremental addresses.

There should be no impact on anyone not using more than 255 nodes as before. One required change for those wanting to use addresses larger than 1 byte (255) is they need to use uint16_t variables in their sketch. I’m looking forward for constructive feedback and any bug reports at this time. The ListenMode part of the code (which is experimental) was left untouched and it only supports 8bit addressing as before.

Here is the latest RFM69 packet header structure including a breakdown of the CTL byte:

Other important updates:

  • The BROADCAST address was changed to 0 (from 255 previously).
  • The previous RFM69 v1.2.0 included SPI Transactions, another important change to the library.
  • After it was brought up in the forum that Moteino AVR sketches compile to larger sketch size than Arduino, I recently released a new  AVR Moteino Boards definition (based on Arduino 1.6.23 AVR boards) which results in significant compiled size reductions for all Moteino sketches, please give this a try. In Arduino IDE’s Boards Manager you should be able to easily upgrade to the latest version.

CurrentRanger review & availability

Sunday morning I woke to an unusual amount of email asking about the CurrentRanger availability. And the limited stock was all gone, so I knew something happened.

I was then pointed to to Andreas Spiess’s latest video:

Thanks to Andreas for posting a thorough review of the CurrentRanger, I especially appreciate how he was able to quickly tune the code to his own needs and customized the CurrentRanger to behave the way he wanted.

He also kindly posted a 3D printable model of the case he shows in the video, some folks already printed it and it looks great! Find it on thingiverse here. The stock case needs some modification to fit the green terminal and access to the USB case, as well as allow mounting the buzzer. If you have a 3D printer you can print this case and keep the black stock case for another project, thanks Andreas!

Availability

I am working to get more units in stock this week. There are a lot of moving parts to making this product. First a large BOM and a complex assembly and testing procedure. I want to ensure to the best of my ability that each unit is able to deliver what it claims.

Some components like the banana jack terminals and OLEDs come from the place we hate to love, China. I’m currently waiting for the small banana jack terminals and OLEDs. However I have a surplus of GOLD terminals, and could replace the small terminals with those if some folks are interested – let me know!

Pricing

I can understand concerns about the cost especially for hobbyists or students. Let me reiterate what I’ve already mentioned in the forum and to others. Here’s the TL;DR of that:

I always tend to design something that I would first and foremost use. While not a high end product, this is not a toy either. I spent over a year – among other things of course 🙂 – developing this product and I put a lot of thought into pricing it before release. I’m not interested in selling high volumes at razor thin returns on my investment and my effort. I prefer fewer sales for people that can appreciate it and not abuse it. I think it was priced fairly, given the high cost of the BOM and the complexity and time it takes to make and test, it really is probably the most complex thing I ever made both in hardware and software. Also, it is not much more expensive than the uCurrent, I am sure anyone reasonable who understands the differences and the set of extra features (perhaps watched the video above for some contrasts), can appreciate all that for just $30 more. The uCurrent is always a great option for those needing accurate current measurements at a lower price.

Romania: Happy 100th birthday!

My readers may not know that I hold dual citizenship, and was born and spent my first 19 years of life in Oradea – a beautiful city at the foot of the pittoresque Carpathian Apuseni mountains and in the western part of Transylvania – the legendary land of Dracula’s castle.

Today’s blog is not about electronics or making stuff, but about what in great part shaped me and perhaps dear reader – what made some things great and significant for you as well. Maybe you’re not a history enthusiast, but maybe you’re a woman engineer, or perhaps you stepped foot in an airplane that took off on its own, used a fountain pen, or depend on insulin, or maybe you just own a LowPowerLab product that has bettered your life in some way – you might be surprised to find out which nation gave the world their inventors.

What’s the big deal?

Today – December 1st 2018 – marks a special anniversary for me as a Romanian diasporan. Exactly 100 years ago, a centuries long ambition and dream was fulfilled, the Great Union was decreed and the Romanian Kingdom became what was then the largest territory that was ever united within its borders and under its name. It integrated present day’s Moldova, and several other smaller Romanian speaking territories, that are today part of Ukraine and Bulgaria.

It then took it’s first steps towards a sovereign democratic nation with a new 1923 constitution sponsored by King Ferdinand I that was deemed the most fundamentally democratic constitution in Europe at that time.

In the greater European context, the end of World War I also marked the end of four empires (Austro-Hungarian, Ottoman, German and Tsarist) and created a spectacular reconfiguration of borders in which a few other nations were reborn (Poland) or created altogether (Czechoslovakia, Yugoslavia).

Since then, a tumultuous century marked several major dramatic events that shaped Romania into what it is today.

  • the conclusion of The Second World War and the soviet occupation triggered the massive territorial losses that can be seen in the map above, and the end to Romania’s constitutional monarchy.
  • the 50+ years of communist rule that followed remains a dark period with profound impacts in every aspect of life, culture and faith. The socialist atheist state attempted to rewrite history, reshape its subjects into “a new type of man“, and imprisoned, tortured and murdered many of those who were perceived as political opponents or dared publicly object to their dialectical materialist agenda – including some of the key personalities that organized this very Great Union event we’re commemorating today:
    • Iuliu Maniu – a great man and political personality of the communist opposition
    • cardinal Iuliu Hossu – Greek Catholic bishop, and the man who officially read the Union’s declaration this day in 1918. In 1948 he refused a forced merger with the corrupted Orthodox Church which bent to the will of the red terror. He paid with his freedom and died in captivity. Interesting fact: he is related to me – my paternal grandfather’s sister was married to his brother Traian Hossu. Among many others, Alexandru Rusu, also a Greek Catholic bishop, and my great grandfather’s brother, also died in prison for the same opposition to become instruments of the stalinist deep state.
  • 1989 brought a sweeping change across the eastern block and the bloodiest revolution behind the Iron Curtain – it toppled the totalitarian regime of Nicolae Ceaușescu and marked Romania’s first steps back towards democracy in modern time. I was just 5 years old at the time.
  • with great expectation joined the NATO alliance in 2004, and in 2007 the European Union.

What’s a Romanian guy doing in the USA?

I ended up coming to America as a student in 2003, my F1-visa dubbing me an “non-resident alien”, a far cry from a proud patriot of the only Eastern European nation with Latin origins, but at least not an invader on a boat or migrant caravan seeking wellfare. Being an international student meant I had to be enrolled full time and pay out-of-state tuition (think 2X+). I covered that and my personal expenses doing occasional construction helper jobs like remodeling, digging plumbing trenches in Arizona’s 115°F hot summers, and laying tile in estates around Phoenix metro – perhaps not my parent’s dream job who had other aspirations for my university years – but all this to avoid being a burden to my beloved aunt (and only relative), who offered me free housing this entire period. By dramatic turning of events and nothing short of a divine miracle, my family was granted a rare type of immigrant visa in 2006 and I was narrowly qualified in that transition just before turning 21, receiving a Green Card which was a huge relief to my financial burdens as a foreign student. I was now a “permanent resident alien”, a clear upgrade from my previous status and a mere 5-year wait for the final upgrade to citizenship. I was now able to find my first programmer job at a real company, making at least double hourly while sitting in an air conditioned office, another miracle for me. The rest is history – I achieved my goal and graduated from ASU without student loans or debt, married to another Romanian immigrant, became the father of two wonderful boys, and last but not least created LowPowerLab and lots of happy customers along the way. Looking back at the footsteps of my immigrant journey, God was good to me and held my hand all all these years, carried me when I couldn’t walk.

Will I ever move back to the country that gave me (and you) so much?
I can’t say yes, but I sure can’t say no either. When I was 19, it was easy to pack everything I owned (not much – mostly clothes, books, small personal effects) in my 2 free allowed suitcases, take my personal $200 savings and hop a plane to a new place with better prospects – if all failed I could return to my previously well planned endeavor to become an engineer in my own country. Things look a bit different now – if not larger suitcases, it would take a lot more effort and arrangements to move a whole family to a new system and way of life and start everything from scratch – at least not without a major reason.

I wrote this up because it’s a centenary that won’t happen again in my lifetime, and thus an excuse to break and pay some tribute to history, brag about my heritage and share a bit of how I got where I am. If you’ve read this far, I thank you and I hope my story and my native country’s history inspired you and made you a bit richer in some small way. I will leave you with a few pretty images that present Romania and some of her humble achievements on this special anniversary, and the sounds of its unique language (which unfortunately were left untranslated by the creator), and hence perhaps one day you may want to uncover Romania on your own, just as Rick Steves or Peter Hurley did recently.

 

Portable RaspberryPi IP Camera Monitor

RaspberryPi projects are always lots of fun, and when combine it with displays, 3D printing and Moteinos, the result can only be awesome!

I had an older RaspberryPi and a touch display, and I wanted a portable IP camera monitor which I can mount or move anywhere. You can build a similar portable display, it will run on battery up to a few hours depending battery capacity – and it will automatically safely shutdown if the battery dips below a programmable voltage. It’s a nice convenient way to monitor your smart home interface, or an IP camera, weather or whatever else.

I had to find a suitable case and this adafruit one was almost perfect as-is. But who uses a Pi with a hard power switch? I wanted to use my own hardware (MightyBoost + Moteino) for battery power and backup, as well as a convenient power and reset button, I had to mod the case and add some holes to make everything fit. I also added some tripod mounts so I could mount this on a tripod or hang it from a shelf or a cabinet. Read on for the build details. I also put together a guide that will be maintained with any future changes to this project.

Continue reading

CurrentRanger: auto-ranging current meter

I’ve always wanted a fast auto-ranging low-burden voltage current meter. You may find expensive high end bench meters which can auto-range they may be slow or lack the bandwidth to capture fast dynamic loads that go through several orders of magnitude of current consumption. Most multimeters also have a large burden voltage, which means their internal current shunts can cause your DUT to see a very significant voltage drop.

I own a µCurrent GOLD from EEVBlog which is great in that it has the precision and bandwidth to capture fast current transients, but it’s a simple manual device that cannot auto-range and unfortunately it’s really noisy in the nA range where it picks up mains noise and it’s unusable without an extra cap on the input (I thought mine was broken but this problem is also reported in the EEVBlog forum here and here). It’s useful when you know your DUT is going to stay in 1 range or if you can predict when your DUT might wake up from deep sleep and manually range just before that happens but it’s a guessing game. Hence the mechanical switches get a lot of abuse and wear, add output noise during switching, and introduce some contact resistance with noticeable effect in the mA range.

Looking around, I couldn’t find much else in terms of affordable fast auto-ranging and highly precise ammeters. So over year ago I started to design my own version of a current meter that has the precision and bandwidth, can auto-range, and has some extra nice-to-have features like:

  • Low pass filter for smooth oscilloscope tracing
  • Unidirectional measurement mode by default for maximum measurement range
  • Standalone OLED display
  • Bluetooth logging would be nice
  • Buzzer for tactile feedback (and why not also play some Beethoven on power-on)
  • Auto-power-off to spare the battery, oh how I craved this simple feature
  • LiPo powered, rechargeable, easily re-programmable – wouldn’t all that be great?
  • Great value vs. features without breaking the bank

It was a bit ambitious and immediately became obvious that this needs to be digitally controlled by a microcontroller to do all that. Five prototype revisions and a year later I think the result is finally ready for release. So I’m pleased to introduce the CurrentRanger, click here for full specifications and user guide.

As a side story – out of the birth pains of the CurrentRanger, resulted the Moteino M0 which uses the same SAMD21 ARM processor that controls the CurrentRanger.

The CurrentRanger is now available in the shop. Please let me know if you did something interesting with this meter. It’s certainly a complex device with a large BOM and lots of parameters to test. With your help I think it can be made even better in so many ways. As resources/code/new features become available they will be added in the CurrentRanger Guide.

Enjoy!

New products: M0 Breakout boards

There are now a few more breakouts available in the shop. With the release of MoteinoM0 I also wanted to make available a few boards specifically designed to mount flat on the M0 PCB for a super compact wireless sensor platform.

The new boards include a BME280 Breakout featuring the same sensor as on the previous WeatherShield

And there is an all new LSM9DS1 9DoF IMU Breakout featuring 3 sensors in 1 compact chip (acelerometer+gyroscope+magnetometer):

 

And here’s a mini SD-Card Logger Breakout featuring a low power switch to keep that hungry SDCard off when not in use.

You don’t even need to use pins at all, just solder the castellated side mounts (aka half holes). You may later desolder these with hot-air or a wide chisel tip on your solder iron. You can of course use these on a breadboard or with any other Arduino/dev board. You can also stack all these on a single MoteinoM0 if you’d like, just need to get a little creative with stacking them to keep pins connected correctly:

Details and sample code were added to the guide page.

Weekend Project: Wireless Microscope Light Ring

I have an Bausch & Lomb optical Stereo-Zoom (SZ4) microscope which is a great instrument and not a luxury when you do a lot of SMD prototyping and repairs. The light solution for this was a simple piece of white LED strip powered from a 12V adapter, worked well for over 6 years and I thought an upgrade to this will make a nice weekend project.

To really make this nice and portable it had to be very compact, wireless power from a rechargeable Lithium-Polymer battery. But how can this be powered from 3-4V when the LED strips require 12V?

Watch the details in the video below, along with a demo of laser-cut SMD stencils and complete hand assembly and test of the new light ring.

The before and after shots:


The design files are over at Github. Are you planning to make one or add more features? Did you learn something new from this video? Have a question or other suggestions? Let me know in the comments!

Transceivers In-Depth Overview

I prepared a complementary video to cover all the basics of the LoRa and RFM69 transceivers used on Moteino boards. I concentrated on the most common topics I’ve seen show up in the forum and in support emails. This should be a good primer for those getting started with using sub-Ghz transceivers with Arduino or Moteino or even other compatible boards, since most principles apply the same way. Feedback and comments are welcome.

Pick and Place Nozzle Demagnetization

Since my LE40-V Pick & Place machine’s factory PCB holder is difficult to use, has no support, and is not well suited for V-scored panels, I had to get creative and use neodymium magnets on a metal plate to hold the PCB panels steady during placement.

This makeshift solution got me in trouble and it’s the second time I managed to get a neodymium magnet on the tip of an expensive SMD pickup tool. In my last post I showed how I upgraded from magnets to 3D printed magnetic brackets to make the PCB holding task easier and avoid this problem. Otherwise this would also not be a problem if I were’n using magnets, yes I know. Or if these tools were machined from a non-magnetic tooling alloy, such as what’s used in these excellent $5 CHP 3-SA tweezers. Don’t ferromagnetic metals get magnetized over time from extensive use anyway? I think it’s common sense to use a quality non-magnetic tooling alloy in such expensive tools which will last the life of the machine and won’t develop such problems.

The workaround I found to work and save the magnetized nozzles is shown in the video below. This method should work for other types of tooling.