Author Topic: Moteino + Spark Core gateway instead of Pi - Solar Powered Weather Station  (Read 20714 times)

syrinxtech

  • Sr. Member
  • ****
  • Posts: 306
  • Country: us
    • Syrinx Technologies
Re: Moteino + Spark Core gateway instead of Pi - Solar Powered Weather Station
« Reply #75 on: September 04, 2016, 06:44:44 AM »
Started getting too many SMS messages during testing so I added email via the great "mailgun" service.  Now I only get SMS messages when the waterleak detector goes off or one of the motion sensors fires.

Added an in-memory node database so I can perform two checks at midnight:
1. See if any sensor node hasn't checked in during the past 48 hours.
2. For battery operated nodes, send an email if any battery level is getting low.

Getting ready to add another in-memory structure holding the highs/lows of key weather metrics on a daily basis.  I'll put this as another screen on Freeboard.

xavier

  • Jr. Member
  • **
  • Posts: 65
  • Country: us
Re: Moteino + Spark Core gateway instead of Pi - Solar Powered Weather Station
« Reply #76 on: September 08, 2016, 02:07:36 PM »
hey silly question: do you store all data points locally? why not push everything to a server and have some cron jobs check the data and calculate daily, weekly averages?

syrinxtech

  • Sr. Member
  • ****
  • Posts: 306
  • Country: us
    • Syrinx Technologies
Re: Moteino + Spark Core gateway instead of Pi - Solar Powered Weather Station
« Reply #77 on: September 08, 2016, 02:57:03 PM »
xavier,

Not a silly question at all.  Right now I'm technically "storing" the data points.  For my weather-related data, it goes over the serial connection to a Particle Photon and up to the cloud where it is displayed via Freeboard.  I also have an SD card on-board if I want to do some local storage.  Down the road I plan on incorporating a MySQL server that will be feed via webhooks from the Photon.

On my list of TODO's is a small local file containing the daily highs/lows/averages that is kept on a daily basis.  At midnight the base station will run a job to zero out the counters and start fresh the next day.  Freeboard will be used to show the results of that daily file.  At some point I will probably write that daily information out to a longer term database like the weather data.

syrinxtech

  • Sr. Member
  • ****
  • Posts: 306
  • Country: us
    • Syrinx Technologies
Re: Moteino + Spark Core gateway instead of Pi - Solar Powered Weather Station
« Reply #78 on: September 11, 2016, 10:28:49 AM »
Added a "daily recap database", (in memory), that holds the previous 24 hours mins/maxes for temp, hum, pressure, wind, rain, etc.  This resets at midnight.

Getting ready to write code to produce two reports to run at noon each day.

1. Those nodes that haven't checked in for at least 48 hours.
2. Battery-operated nodes that have a battery level less than a preset value.

Including the various modules where I've broken out code to make things easier to find and modify, I guess there is about 1,200 lines of code for the base station.  There is least another 1,000 lines combined for the various sensors and display units.  If the 328p was only multi-threaded!!



xavier

  • Jr. Member
  • **
  • Posts: 65
  • Country: us
Re: Moteino + Spark Core gateway instead of Pi - Solar Powered Weather Station
« Reply #79 on: September 16, 2016, 12:34:51 PM »
hey syrinxtech: how long do you measure wind speed before returning the data? I'm currently sampling for 3s and my gateway requests data every minute, probably overkill since I now have 81K entry points (2 months of data) - the pressure data is interesting though and by calculating the rate of change I can "almost" say if the fog comes in or just clouds!

I missed the fact you were using Freeboard - I was thinking switching my bananaPI to a ESP8266 (just saw the post from Xose as well, very interesting and detailed blog!) but I'm not sure it would improve anything...I'll just wait for the ESP32 so I can configure the gateway using my iPhone (that would enable anyone to pretty much grab a gateway and some nodes and configure everything without coding)

I like the fact I can run socket.io on the PI and have web client anywhere be able to request data, I suppose mqtt would work as well but I looked at the adafruit samples and it's heavy (or maybe I need to spend more time looking at the code)

syrinxtech

  • Sr. Member
  • ****
  • Posts: 306
  • Country: us
    • Syrinx Technologies
Re: Moteino + Spark Core gateway instead of Pi - Solar Powered Weather Station
« Reply #80 on: September 16, 2016, 09:40:42 PM »
xavier,

I'm not actually sure how long it measures.  I'm using the WeatherRack from SwitchDoc Labs.  The code is on the page if you're interested in looking it over:

http://www.switchdoc.com/weatherrack-weather-sensors/

I'm in a sampling loop between 3 sets of data:  1) Temp/hum/press, 2) Electrical - 6 different values from the INA3221, and 3) Wind/Rain. 

I really like Freeboard, in fact I just added a mobile version of the normal weather page that doesn't use any fancy graphics so everything fits onto the screen.  I just finished the daily statistics page that I added to the mobile version.

I am really interested in playing more with mqtt myself. 

Here's the URL for the Freeboard "mobile" version with daily recap:

https://freeboard.io/board/5epTjs


xavier

  • Jr. Member
  • **
  • Posts: 65
  • Country: us
Re: Moteino + Spark Core gateway instead of Pi - Solar Powered Weather Station
« Reply #81 on: September 22, 2016, 08:15:31 PM »
is that last URL you sent using data from your weather rack setup? (100% humidity seems high though :-)
I see the data changing almost RT in certain cases... I guess I'll check their code.

My thinking w using node and socket.io was to use that and an external server to get data every 3 sec from the weather station when the iPhone app is used. I guess I should start updating my iPhone app...

syrinxtech

  • Sr. Member
  • ****
  • Posts: 306
  • Country: us
    • Syrinx Technologies
Re: Moteino + Spark Core gateway instead of Pi - Solar Powered Weather Station
« Reply #82 on: September 22, 2016, 10:48:21 PM »
Yes, that last link is my 2nd Freeboard setup.  I created it without using any gauges or fancy icons so it will fit on an Amazon Fire in landscape mode.  I get all of the weather and electrical data plus a 24-hour recap of the highs/lows.

We frequently see 100% humidity around here.  Particularly when it has recently rained or there is a lot of dew in the morning.

Some of the measurements are sampled at 30 seconds while others, like the wind, are sampled at 3 second intervals.  Freeboard reads the data from Particle.IO in JSON format.  The Particle Photon connected serially to the Moteino receives all of the data and uploads it to be read by Freeboard.  The only limitation at the moment is a total of 20 variables you can monitor.  In some cases I've had to make a string and combine 2 or 3 measurements instead of reporting them individually.

The wind/rain component I'm using are setup to use interrupts for wind and rain changes.  Each .01" of rain causes an interrupt to count the rain and the anenometer also uses an interrupt.  The wind direction is measured using reed switches.

syrinxtech

  • Sr. Member
  • ****
  • Posts: 306
  • Country: us
    • Syrinx Technologies
Re: Moteino + Spark Core gateway instead of Pi - Solar Powered Weather Station
« Reply #83 on: February 07, 2017, 10:11:28 AM »
Made a couple of improvements in timekeeping, both on the Spark Photon and the Moteino.

Added code to adjust for DST.  My normal is EST (-5) and twice a year it changes between EST and EDT.  The offset is one hour.  The code basically adjusts for the difference and sets the time accordingly.  The Photon gets its time via NTP from the Internet and the Moteino keeps time using a RTC board.  Before this, I had to recompile programs on both platforms in order to keep the time correct, especially since all texts and emails originate from the Photon and when the time changed the timestamp on the emails was wrong.


syrinxtech

  • Sr. Member
  • ****
  • Posts: 306
  • Country: us
    • Syrinx Technologies
Finally got around to making an improvement on the SD card logging routines.  I added a couple of control functions and now I can remotely dump the contents of the SD card over the radio to a client.  This allows me to dump the log to a file using CoolTerm without having to physically remove the SD card, copy off the file, and replace it.  I borrowed this code from another project I just recently finished and in the other project I also have the ability to delete the file so I can always start over fresh without retrieving the card. 

I wrote a "universal" client, allowing me to connect to these two different applications running on two different basestations.  Each has their own radio frequency and node ID's.  With a little input from Serial.available I dynamically map the radio to correct configuration and deliver a unique structure of information to request the SD card file dump, or delete.

syrinxtech

  • Sr. Member
  • ****
  • Posts: 306
  • Country: us
    • Syrinx Technologies
And since I do security auditing for a living, I thought I would add some additional layers of security.

On top of the normal network ID and encryption key needed, I added two additional security features when downloading the SD card logfile contents OTA:

1.  I added a "magic cookie" in the request structure so if you don't have the right bytes in the right fields you don't get any data.
2.  I restricted the source node ID to a single device so you have to make requests from the mgmt station.


rwb

  • Newbie
  • *
  • Posts: 6
  • Country: us
@Syrinxtech 

I saw this thread and wanted to check in and let you know that we have the Radio Head library working on the Particle Photon devices now. So you can hook up an RFM95 module directly to the Photon now and it works great.


You can find more info on the Radio Head library working directly with the Particle Photon in this thread: https://community.particle.io/t/particle-photon-electron-rfn95w-long-range-radio/22439/20?u=rwb

I'm looking to create a network like pictured below where only the Server Photon has WiFi access.


I'm looking for help on how to format variable/ sensor data using the Radio Head RFM95 Reliable Datagram code when sending to from the Client radios to the Photon server radio.

Once the data is received via the Server Photon I need to update the proper variables with the new sensor data.

I see you have done this already with the RF69 radios but I'm not sure how similar that is to the RFM95 radio send data format.

Let me know what you think if you have time! Thanks! ;)

« Last Edit: May 26, 2017, 04:13:42 PM by rwb »

syrinxtech

  • Sr. Member
  • ****
  • Posts: 306
  • Country: us
    • Syrinx Technologies
@rwb,

Thanks for the heads up on the Photon.

Second, in your "Particle Lora" picture, you wouldn't necessarily need the Photons on the edge.  You might be able to save some money by just purchasing a LowPowerLabs Moteino with the appropriate radio.  Seems a waste to have wireless connectivity provided by the Photons and then use the RF radios to transmit your sensor data.  Just a thought.

Third, I've worked with the RadioHead code in the past and there really isn't a difference in the mechanics of sending a packet over their API versus Felix's RFM69 code.  You create the appropriate structure containing the necessary data and call some form of a "send" function.  Creating the best structure for the wireless data depends on exactly what you're sending and how you plan on representing the data.  There have been many discussions on this forum about transmitting chars, floats, ints, long ints, etc., that you might want to search for and take advantage of the collective wisdom of this group.

raenrfm

  • Newbie
  • *
  • Posts: 20
Re: Moteino + Spark Core gateway instead of Pi - Solar Powered Weather Station
« Reply #88 on: September 11, 2017, 04:56:47 PM »
I'm basically doing the same setup right now, accept I'm storing my data locally on an MQTT server, then on my PI I'm running WeeWX (weather software), you should really look at it, instead of re-inventing the wheel.  That package has everything you need to archive data (SQL based) and do trending etc...it also does calculations for you.  If you set it up with realtime gauges it generates a web page that you can display on any device (wall mounted tablet??) and the gauges look like old school weather gauges.  There is also a very good community of weather nerds like myself that do all kinds of stuff like this and have their own builds.

Right now my MQTT server is running on a PC, but I hate having it on all the time, so my goal is to create a PI cluster and run a Docker swam and have MQTT and Weewx, rednode and whatever else I can think of running on it.  Want to have it on battery backup as well in case of an outage so it will never stop.  My base station will also have a lipo battery for the same reason.  I'm using the same Adafruit solar charger and it's the bomb, works amazing as long as you feed it a 6V panel.

Go check out Weewx, you'll thank me later.