Hi Felix & All,
I'm still in the process of implementing my PiGateway. I'm waiting on a MightyHat that it's currently flying it's way to Australia :)
While I'm waiting, I've been playing with the RaspberryPi 3B+ and getting things setup as far as I can. I've managed to successfully setup the Gateway software and have a no-ip account to access the Gateway from the internet and it all works well so far. So functionally it's all good, I just don't have a datasource to read yet.
What I'd like to do is load data into a dummy node. The data would be generated by the RasPi containing the internet upload / download speeds obtained by the speedtest-CLI package that I installed.
As I'm just getting started, I was wondering if someone could point me to the relevant area to learn about creating a dummy script to record the internet speed run by crontab for sending to the gateway listener process on the raspberry?
I (probably incorrectly) figure I should be able to do this while I don't have a mightyhat installed. Or am I out of luck until I actually connect something to the Serial port?
Changed my mind and decided to try and get a moteino connected to the gateway. After a bit of bumbling, I realized I forgot to do the step that would apply to a Pi3B+ where you requested to edit the /boot/config.txt with
enable_uart=1
dtoverlay=pi3-disable-bt
As soon as I fixed that, everything mostly came up Milhouse! Although, I found that I'm getting a bit excessive NOK responses from the transmitter after it tries to do it's send. I did the check of the gateway and it's getting the message and sending the ACK but the sender doesn't seem to be able to reliably see the ACK.
I'm using both the standard PiGateway.ino (https://github.com/LowPowerLab/RFM69/blob/master/Examples/PiGateway/PiGateway.ino) script and the SonarMote_DistanceTracker.ino (https://github.com/LowPowerLab/RFM69/blob/master/Examples/SonarMote/SonarMote_DistanceTracker/SonarMote_DistanceTracker.ino) with only changes are commenting out the IS_RFM69HW_HCW and changing the frequency to 433Mhz. Would having the SERIAL_EN enabled cause the transmitter to be too busy to respond to an ACK? I'm kind of grasping at straws as to why I'd see anything from 2 to 15 ACK:NOK before an ACK:OK
Now that I have a functional gateway, is there anything I can do to get the raspberry pi to communicate with its own serial port to send a dummy node packet to get the internet speed logging within the gateway interface?
If you have a W/CW then commenting out the #IS_RFM69HW_HCW setting is correct.
Matching settings correctly is crucial to getting a reliable link. Use default settings to start with, match frequency, network, and have separate IDs (1 for gateway), and the HW setting on each node.
Make sure antennas are well soldered, and they do not live in metal boxes.
Quote from: wile1411 on October 22, 2018, 12:24:35 AM
Now that I have a functional gateway, is there anything I can do to get the raspberry pi to communicate with its own serial port to send a dummy node packet to get the internet speed logging within the gateway interface?
Not sure what you mean by "anything to get the internet speed logging within the gateway interface". How fast your internet is? No .. there is no code for that. Maybe you can write something up to poll an API or speedtest.net and report that in a dummy node somehow.
Quote from: Felix on October 22, 2018, 10:53:01 AM
Not sure what you mean by "anything to get the internet speed logging within the gateway interface". How fast your internet is? No .. there is no code for that. Maybe you can write something up to poll an API or speedtest.net and report that in a dummy node somehow.
I've got the code to do the speedtest process metrics, so that's no problem. As I understand it, it's easy enough to get an external moteino to send data to the gateway moteino which then passes it to the RasPi serial port to log using the Gateway software I installed.
What I'd like to do is have the raspberry itself be a data generator and send my speedtest data directly to the gateway software. I know I need to setup the metrics that I want to add into the metrics.js file and I think I can work that out based on the examples you've provided. Is it possible to have a py script (or something) connect to the listening gateway app and send the appropriately formatted data directly to the socket listener? Or is the only way to get data into the system is via the moteino that talks to the gateway?
Quote from: Felix on October 22, 2018, 10:53:01 AM
If you have a W/CW then commenting out the #IS_RFM69HW_HCW setting is correct.
Matching settings correctly is crucial to getting a reliable link. Use default settings to start with, match frequency, network, and have separate IDs (1 for gateway), and the HW setting on each node.
Make sure antennas are well soldered, and they do not live in metal boxes.
There's no boxes as yet and yes I'm using the old FRM69W modules on all my moteinos. I've attached a few images to help explain my testing setup. I'm using the breadboard to connect the raspberry to the gateway moteino as I don't have any female-female cables and the test moteino already has the 90 degree header soldered on. Here is what's on the testing bench with the distance between the Gateway and the sonarmote about 80cm.
Transmitter SonarMote// Sample RFM69 sender/node sketch for the SonarMote - Distance tracker
// Can be used for inventory control - ex to measure distance in a multi lane cigarette pack rack
// http://lowpowerlab.com/sonarmote
// Ultrasonic sensor (HC-SR04) connected to D6 (Trig), D7 (Echo), and power enabled through D5
// This sketch sleeps the Moteino and sensor most of the time. It wakes up every few seconds to take
// a distance reading. If it detects an approaching object (car) it increases the sampling rate
// and starts lighting up the LED (from green to yellow to red to blinking red). Once there is no more
// motion the LED is turned off and the cycle is back to a few seconds in between sensor reads.
// Button is connected on D3. Holding the button for a few seconds enters the "red zone adjust" mode (RZA).
// By default the red zone limit is at 25cm (LED turns RED below this and starts blinking faster and faster).
// In RZA, readings are taken for 5 seconds. In this time you have the chance to set a new red zone limit.
// Valid new red zone readings are between the RED__LIMIT_UPPER (default 25cm) and MAX_ADJUST_DISTANCE (cm).
// In RZA mode the BLU Led blinks fast to indicate new red limit distance. It blinks slow if the readings are invalid
// If desired this value could be saved to EEPROM to persist if unit is turned off
// Get the RFM69 at: https://github.com/LowPowerLab/
// Make sure you adjust the settings in the configuration section below !!!
// **********************************************************************************
// Copyright Felix Rusu 2016, http://www.LowPowerLab.com/contact
// **********************************************************************************
// License
// **********************************************************************************
// This program is free software; you can redistribute it
// and/or modify it under the terms of the GNU General
// Public License as published by the Free Software
// Foundation; either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will
// be useful, but WITHOUT ANY WARRANTY; without even the
// implied warranty of MERCHANTABILITY or FITNESS FOR A
// PARTICULAR PURPOSE. See the GNU General Public
// License for more details.
//
// Licence can be viewed at
// http://www.gnu.org/licenses/gpl-3.0.txt
//
// Please maintain this license information along with authorship
// and copyright notices in any redistribution of this code
// **********************************************************************************
#include <RFM69.h> //get it here: https://www.github.com/lowpowerlab/rfm69
#include <SPI.h> //included with Arduino IDE (www.arduino.cc)
#include <LowPower.h> //get library from: https://github.com/lowpowerlab/lowpower
//writeup here: http://www.rocketscream.com/blog/2011/07/04/lightweight-low-power-arduino-library/
//*********************************************************************************************
//************ IMPORTANT SETTINGS - YOU MUST CHANGE/CONFIGURE TO FIT YOUR HARDWARE *************
//*********************************************************************************************
#define NODEID 22 //unique for each node on same network
#define GATEWAYID 1 //node Id of the receiver we are sending data to
#define NETWORKID 200 //the same on all nodes that talk to each other including this node and the gateway
#define FREQUENCY RF69_433MHZ //others: RF69_433MHZ, RF69_868MHZ (this must match the RFM69 freq you have on your Moteino)
// #define IS_RFM69HW_HCW //uncomment only for RFM69HW/HCW! Leave out if you have RFM69W/CW!
#define ENCRYPTKEY "sampleEncryptKey" //exactly the same 16 characters/bytes on all nodes!
#define SENDLOOPS 80 //default:80 //if no message was sent for this many sleep loops/cycles, then force a send
#define READ_SAMPLES 3
#define HISTERESIS 2 //(cm) only send a message when new reading is this many centimeters different
#define DIST_READ_LOOPS 2 //read distance every this many sleeping loops (ie if sleep time is 8s then 2 loops => a read occurs every 16s)
//*********************************************************************************************
//#define BUZZER_ENABLE //uncomment this line if you have the BUZZER soldered and want the sketch to make sounds
#define SERIAL_EN //uncomment if you want serial debugging output
//*********************************************************************************************
#define SLEEP_FASTEST SLEEP_15MS
#define SLEEP_FAST SLEEP_250MS
#define SLEEP_SEC SLEEP_1S
#define SLEEP_LONG SLEEP_2S
#define SLEEP_LONGER SLEEP_4S
#define SLEEP_LONGEST SLEEP_8S
period_t sleepTime = SLEEP_LONG; //period_t is an enum type defined in the LowPower library (LowPower.h)
//*********************************************************************************************
#ifdef __AVR_ATmega1284P__
#define LED 15 // Moteino MEGAs have LEDs on D15
#define FLASH_SS 23
#else
#define LED 9 // Moteinos have LEDs on D9
#define FLASH_SS 8
#endif
#define TRIG 6 // digital pin wired to TRIG pin of ultrasonic sensor
#define ECHO 7 // digital pin wired to ECHO pin of ultrasonic sensor
#define SENSOR_EN 5 // digital pin that enables power to ultrasonic sensor
#define BUZZER 4 // digital pin that is connected to onboard buzzer
#define MAX_DISTANCE 150 // maximum valid distance
#define MIN_DISTANCE 1.5 // minimum valid distance
#define MAX_ADJUST_DISTANCE (MAX_DISTANCE-GRN_LIMIT_UPPER) //this is the amount by which the RED_LIMIT_UPPER can by increased
#ifdef SERIAL_EN
#define SERIAL_BAUD 115200
#define DEBUG(input) {Serial.print(input);}
#define DEBUGln(input) {Serial.println(input);}
#define SERIALFLUSH() {Serial.flush();}
#else
#define DEBUG(input);
#define DEBUGln(input);
#define SERIALFLUSH();
#endif
#define BATT_MONITOR A7 // Sense VBAT_COND signal (when powered externally should read ~3.25v/3.3v (1000-1023), when external power is cutoff it should start reading around 2.85v/3.3v * 1023 ~= 883 (ratio given by 10k+4.7K divider from VBAT_COND = 1.47 multiplier)
#define BATT_READ_LOOPS SENDLOOPS // read and report battery voltage every this many sleep cycles (ex 30cycles * 8sec sleep = 240sec/4min). For 450 cycles you would get ~1 hour intervals between readings
#define BATT_FORMULA(reading) reading * 0.00322 * 1.475 // >>> fine tune this parameter to match your voltage when fully charged
#define BATT_LOW 3.55
byte sendLen;
byte sendLoops=0;
byte distReadLoops=0;
byte battReadLoops=0;
float distance=0;
float prevDistance=0;
float batteryVolts = 5;
char buff[50]; //this is just an empty string used as a buffer to place the payload for the radio
char* BATstr="BAT:5.00v"; //longest battery voltage reading message = 9chars
char* DISTstr="99999.99cm"; //longest distance reading message = 5chars
float readDistance(byte samples=1); //take 1 samples by default
RFM69 radio;
void setup() {
#ifdef SERIAL_EN
Serial.begin(SERIAL_BAUD); // Open serial monitor at 115200 baud to see ping results.
#endif
radio.initialize(FREQUENCY,NODEID,NETWORKID);
#ifdef IS_RFM69HW_HCW
radio.setHighPower(); //must include this only for RFM69HW/HCW!
#endif
radio.encrypt(ENCRYPTKEY);
//sprintf(buff, "\nTransmitting at %d Mhz...", FREQUENCY==RF69_433MHZ ? 433 : FREQUENCY==RF69_868MHZ ? 868 : 915);
sprintf(buff, "\nTransmitting at %d Mhz, id:%d nid:%d gid:%d", FREQUENCY==RF69_433MHZ ? 433 : FREQUENCY==RF69_868MHZ ? 868 : 915, NODEID, NETWORKID, GATEWAYID);
DEBUG(buff);
for (byte i=0;i<strlen(ENCRYPTKEY);i++) DEBUG(ENCRYPTKEY[i]);
DEBUGln();
radio.sleep();
pinMode(TRIG, OUTPUT);
pinMode(ECHO, INPUT);
pinMode(SENSOR_EN, OUTPUT);
digitalWrite(SENSOR_EN, LOW);
#ifdef BUZZER_ENABLE
pinMode(BUZZER, OUTPUT);
buzzer(50,2,100);
#endif
radio.sendWithRetry(GATEWAYID, "START", 5);
SERIALFLUSH();
readDistance(); //first reading seems to always be low
readBattery();
}
void loop() {
if (battReadLoops--<=0) //only read battery every BATT_READ_LOOPS cycles
{
readBattery();
battReadLoops = BATT_READ_LOOPS-1;
}
if (distReadLoops--<=0)
{
distance = readDistance(READ_SAMPLES);
distReadLoops = DIST_READ_LOOPS-1;
}
float diff = distance - prevDistance;
if ((diff > HISTERESIS || diff < -HISTERESIS) || (sendLoops--<=0)) //only send a new message if the distance has changed more than the HISTERESIS or if sendloops has expired
{
if (distance > MAX_DISTANCE || distance < MIN_DISTANCE)
DISTstr = "0"; // zero, out of range
else dtostrf(distance,3,2, DISTstr);
if (diff > HISTERESIS || diff < -HISTERESIS)
sprintf(buff, "%scm BAT:%s", DISTstr, BATstr); //send both distance and battery readings
else
sprintf(buff, "BAT:%s", BATstr); //distance has not changed significantly so only send last battery reading
sendLen = strlen(buff);
digitalWrite(LED, HIGH);
DEBUG(buff);
if (radio.sendWithRetry(GATEWAYID, buff, sendLen))
{
prevDistance = distance;
DEBUG(" - ACK:OK! RSSI:");
DEBUGln(radio.RSSI);
}
else DEBUGln(" - ACK:NOK...");
digitalWrite(LED, LOW);
sendLoops = SENDLOOPS-1; //reset send loop counter
}
radio.sleep();
SERIALFLUSH();
LowPower.powerDown(sleepTime, ADC_OFF, BOD_OFF); //put microcontroller to sleep to save battery life
}
float readDistance(byte samples)
{
if (samples == 0) samples = 1;
if (samples > 10) samples = 10;
digitalWrite(SENSOR_EN, HIGH);
//need about 60-75ms after power up before HC-SR04 will be usable, so just sleep in the meantime
LowPower.powerDown(SLEEP_60MS, ADC_OFF, BOD_OFF);
LowPower.powerDown(SLEEP_15MS, ADC_OFF, BOD_OFF);
PING();
LowPower.powerDown(SLEEP_15MS, ADC_OFF, BOD_OFF);
unsigned long duration = 0;
for (byte i=0; i<samples; i++)
{
duration += PING();
if (samples > 1) LowPower.powerDown(SLEEP_15MS, ADC_OFF, BOD_OFF);
}
digitalWrite(SENSOR_EN, LOW);
return microsecondsToCentimeters(duration / samples);
}
long PING()
{
digitalWrite(TRIG, LOW);
delayMicroseconds(2);
digitalWrite(TRIG, HIGH);
delayMicroseconds(5);
digitalWrite(TRIG, LOW);
return pulseIn(ECHO, HIGH);
}
void readBattery()
{
unsigned int readings=0;
for (byte i=0; i<5; i++) //take several samples, and average
readings+=analogRead(BATT_MONITOR);
batteryVolts = BATT_FORMULA(readings / 5.0);
dtostrf(batteryVolts,3,2, BATstr); //update the BATStr which gets sent every BATT_CYCLES or along with the MOTION message
if (batteryVolts <= BATT_LOW) BATstr = "LOW";
}
float microsecondsToInches(long microseconds)
{
// According to Parallax's datasheet for the PING))), there are
// 73.746 microseconds per inch (i.e. sound travels at 1130 feet per
// second). This gives the distance travelled by the ping, outbound
// and return, so we divide by 2 to get the distance of the obstacle.
// See: http://www.parallax.com/dl/docs/prod/acc/28015-PING-v1.3.pdf
return microseconds / 74.0 / 2.0f;
}
float microsecondsToCentimeters(long microseconds)
{
// The speed of sound is 340 m/s or 29 microseconds per centimeter.
// The ping travels out and back, so to find the distance of the
// object we take half of the distance travelled.
return (float)microseconds / 29.0f / 2.0f;
}
#ifdef BUZZER_ENABLE
void buzzer(byte soundTime, byte repeats, byte repeatsDelay)
{
for (byte i=0;i<=repeats;i++)
{
tone(BUZZER, 4500); //4500hz makes a nice audible sound from a 3.3v Moteino digital pin
delay(soundTime);
noTone(BUZZER);
if (repeats>0) delay(repeatsDelay);
}
}
#endif
void Blink(byte pin)
{
pinMode(pin, OUTPUT);
digitalWrite(pin, HIGH);
delay(2);
digitalWrite(pin, LOW);
}Gateway Moteino attached to RasPi
// **********************************************************************************************************
// Moteino gateway/base sketch that works with Moteinos equipped with RFM69W/RFM69HW/RFM69CW/RFM69HCW
// This is a basic gateway sketch that receives packets from end node Moteinos, formats them as ASCII strings
// with the end node [ID] and passes them to Pi/host computer via serial port
// (ex: "messageFromNode" from node 123 gets passed to serial as "[123] messageFromNode")
// It also listens to serial messages that should be sent to listening end nodes
// (ex: "123:messageToNode" sends "messageToNode" to node 123)
// Make sure to adjust the settings to match your transceiver settings (frequency, HW etc).
// **********************************************************************************
// Copyright Felix Rusu 2016, http://www.LowPowerLab.com/contact
// **********************************************************************************
// License
// **********************************************************************************
// This program is free software; you can redistribute it
// and/or modify it under the terms of the GNU General
// Public License as published by the Free Software
// Foundation; either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will
// be useful, but WITHOUT ANY WARRANTY; without even the
// implied warranty of MERCHANTABILITY or FITNESS FOR A
// PARTICULAR PURPOSE. See the GNU General Public
// License for more details.
//
// Licence can be viewed at
// http://www.gnu.org/licenses/gpl-3.0.txt
//
// Please maintain this license information along with authorship
// and copyright notices in any redistribution of this code
// **********************************************************************************
#include <RFM69.h> //get it here: https://github.com/lowpowerlab/rfm69
#include <RFM69_ATC.h> //get it here: https://github.com/lowpowerlab/RFM69
#include <RFM69_OTA.h> //get it here: https://github.com/lowpowerlab/RFM69
#include <SPIFlash.h> //get it here: https://github.com/lowpowerlab/spiflash
#include <SPI.h> //included with Arduino IDE (www.arduino.cc)
//****************************************************************************************************************
//**** IMPORTANT RADIO SETTINGS - YOU MUST CHANGE/CONFIGURE TO MATCH YOUR HARDWARE TRANSCEIVER CONFIGURATION! ****
//****************************************************************************************************************
#define NODEID 1 //the ID of this node
#define NETWORKID 200 //the network ID of all nodes this node listens/talks to
#define FREQUENCY RF69_433MHZ //Match this with the version of your Moteino! (others: RF69_433MHZ, RF69_868MHZ)
#define ENCRYPTKEY "sampleEncryptKey" //identical 16 characters/bytes on all nodes, not more not less!
// #define IS_RFM69HW_HCW //uncomment only for RFM69HW/HCW! Leave out if you have RFM69W/CW!
#define ACK_TIME 50 // # of ms to wait for an ack packet
//*****************************************************************************************************************************
#define ENABLE_ATC //comment out this line to disable AUTO TRANSMISSION CONTROL
#define ATC_RSSI -75 //target RSSI for RFM69_ATC (recommended > -80)
//*****************************************************************************************************************************
// Serial baud rate must match your Pi/host computer serial port baud rate!
#define SERIAL_EN //comment out if you don't want any serial verbose output
#define SERIAL_BAUD 19200
//*****************************************************************************************************************************
#ifdef __AVR_ATmega1284P__
#define LED 15 // Moteino MEGAs have LEDs on D15
#define FLASH_SS 23 // and FLASH SS on D23
#else
#define LED 9 // Moteinos have LEDs on D9
#define FLASH_SS 8 // and FLASH SS on D8
#endif
#ifdef SERIAL_EN
#define DEBUG(input) {Serial.print(input); delay(1);}
#define DEBUGln(input) {Serial.println(input); delay(1);}
#else
#define DEBUG(input);
#define DEBUGln(input);
#endif
#ifdef ENABLE_ATC
RFM69_ATC radio;
#else
RFM69 radio;
#endif
SPIFlash flash(FLASH_SS, 0xEF30); //EF30 for 4mbit Windbond FlashMEM chip
void setup() {
Serial.begin(SERIAL_BAUD);
delay(10);
radio.initialize(FREQUENCY,NODEID,NETWORKID);
#ifdef IS_RFM69HW_HCW
radio.setHighPower(); //must include this only for RFM69HW/HCW!
#endif
radio.encrypt(ENCRYPTKEY);
#ifdef ENABLE_ATC
radio.enableAutoPower(ATC_RSSI);
#endif
char buff[50];
sprintf(buff, "\nTransmitting at %d Mhz...", radio.getFrequency()/1000000);
DEBUGln(buff);
if (flash.initialize())
{
DEBUGln("SPI Flash Init OK!");
}
else
{
DEBUGln("SPI FlashMEM not found (is chip onboard?)");
}
}
byte ackCount=0;
byte inputLen=0;
char input[64];
byte buff[61];
String inputstr;
void loop() {
inputLen = readSerialLine(input);
inputstr = String(input);
inputstr.toUpperCase();
if (inputLen > 0)
{
if (inputstr.equals("KEY?"))
{
DEBUG("ENCRYPTKEY:");
DEBUG(ENCRYPTKEY);
}
byte targetId = inputstr.toInt(); //extract ID if any
byte colonIndex = inputstr.indexOf(":"); //find position of first colon
if (targetId > 0)
{
inputstr = inputstr.substring(colonIndex+1); //trim "ID:" if any
}
if (targetId > 0 && targetId != NODEID && targetId != RF69_BROADCAST_ADDR && colonIndex>0 && colonIndex<4 && inputstr.length()>0)
{
inputstr.getBytes(buff, 61);
//DEBUGln((char*)buff);
//DEBUGln(targetId);
//DEBUGln(colonIndex);
if (radio.sendWithRetry(targetId, buff, inputstr.length()))
{
DEBUGln("ACK:OK");
}
else
DEBUGln("ACK:NOK");
}
}
if (radio.receiveDone())
{
int rssi = radio.RSSI;
DEBUG('[');DEBUG(radio.SENDERID);DEBUG("] ");
if (radio.DATALEN > 0)
{
for (byte i = 0; i < radio.DATALEN; i++)
DEBUG((char)radio.DATA[i]);
DEBUG(" [RSSI:");DEBUG(rssi);DEBUG("]");
}
CheckForWirelessHEX(radio, flash, false); //non verbose DEBUG
if (radio.ACKRequested())
{
byte theNodeID = radio.SENDERID;
radio.sendACK();
DEBUG("[ACK-sent]");
}
DEBUGln();
Blink(LED,3);
}
}
void Blink(byte PIN, int DELAY_MS)
{
pinMode(PIN, OUTPUT);
digitalWrite(PIN,HIGH);
delay(DELAY_MS);
digitalWrite(PIN,LOW);
}Actual TestBench showing moteinos. That's a deck of playing cards in front of the Sonarmote that I'll move a bit to see how the responses are.
(https://i.imgur.com/z0q5dr8l.jpg)
(https://i.imgur.com/Vb01BGMl.jpg)
Serial monitor of
SonarMote showing lots of NOK which I can't work out why so many NOK. Do both sides need ATC? The test script only has the gateway with ATC so I tried adding ATC to both sides, but it didn't seem to help.
(https://i.imgur.com/2M05PE9l.png)
I tried to review the serial port with "sudo minicom -b 19200 -o -D /dev/ttyAMA0" and I can see that the gateway moteino get all the messages and I see the Gateway message saying "ACK sent".
The issue seems to be just the SonarMote moteino not seeing the ACK being sent from the gateway.
I'm pretty sure I must be doing something stupid or missing something obvious. I'm going to redo both moteinos and load the Struct_receive & Struct_send and see what results I get with those scripts on their own.
Okie dokie... Struct_receive & Struct_send seems to work fine. See attached file output
I rechecked all my orders with the shop order history and have copy/pasted all transceivers here
Still to arrive:
14/10/2018 - R3 Mighty Hat (Transceiver: RFM69W - 433Mhz)
- R3 Motion Mote (Moteino type: Moteino + RFM69CW-433Mhz)
Have now:
27/09/2018 - R6 Moteino x1 (Transceiver: RFM69CW - 433Mhz)
12/04/2015 - R4 Moteino x4 (Transceiver: RFM69W - 433Mhz)
13/08/2014 - R4 Moteino x4 (Transceiver: RFM69W - 433Mhz)
These should all be 433Mhz according to my orders and should all work with each other as far as I've understood the moteino guide
For all this threads testing, I was using a 2014 white spot for the gateway moteino and a 2015 Gold dot for the sonarmote moteino.
Switching to both white dots give the same result with a lot of NOKs but the Struct_receive & Struct_send seems to work good with AOKs for all messages.
Pictures below - (click for zoomable pic on imgur)
2014
(https://i.imgur.com/Gbs2bapt.jpg) (https://i.imgur.com/Gbs2bap.jpg)
2015
(https://i.imgur.com/M4D4tQqt.jpg) (https://i.imgur.com/M4D4tQq.jpg)
2018
(https://i.imgur.com/UN5P7FRt.jpg) (https://i.imgur.com/UN5P7FR.jpg)
First, there is a guide here showing all the types of transceivers, both with how they are marked, and how they look, this guide should help anyone to identify what they got:
https://lowpowerlab.com/guide/moteino/transceivers/
All yours seem to be the W/CW types, so your settings look correct. Sometimes close proximity can be an issue, I recall from other threads, and I think more at 433mhz but I could not be sure since I have not really observed this effect and I don't use 433mhz.
I would keep antennas straight up from the board, and away from other wiring, but this should not be a huge influence, as I understand your struct examples do work so that's good.
The PiGateway sketch automatically should send ACKs to sender motes.
Sanity check but can you confirm you don't happen to have another node with ID=22, or another gateway with ID=1 on the same network?
It would be easy enough to swap the Moteinos in question and see if anything changes, something you could check and report your findings?
Thanks Felix,
Hahaha :) I can confirm I am sane. I've only got those two (1 and 22) that are even on the network or connected to anything. All 9 moteinos I own are in those pictures and I'm only using two of them.
proximity doesn't seem to be a thing as I still have good results with the structs. I did move the pi and gateway moteino across the room and no change in the results.
Could there be any timing issues I could troubleshoot? Also, is there any sanity checks I can run per board? I'm a bit lost what to do when the send/receive struct scripts work. So if that base level is fine, I'd expect it to be good for other scripts.
Lastly, there was one item I wanted to query in the struct scripts. What was the purpose of the radio.receiveDone() in the sending struct loop? Is there somewhere I can add that to verify the sonar script is not locking up in the sending area?
Timing issue? Not really, nothing I can think of right now.
The examples are given to have a clean start from something that works, they are used by others and I use them myself and there is no problem that I know of.
I haven't looked at all your code but if you add your own code to the sketch, and this is often overlooked, then I cannot vouch for that part, make sure you do not introduce any delays, I often see delay(xxxxxxxx) injected randomly (copy pasted from other sites for other sensors etc).
Why not try the Gateway and Node sketches. Those are barebones sketches and the node sends every 200ms, the gateway asks for an ACK every third packet.
Thanks for the suggestions for the node and gateway.
Found one Moteino that as the gateway would fail to get the ACK back to the node. I'm going to reverse all the tests so each mote has a turn as both the gateway & node, but it looks like one
transceiver might be damaged to the point where it is shown getting "nothing..." a few times for those scripts.
Catch is, not matter which motes I check, all combinations once loaded with the PiGateway & Sonarmote scripts (with 433 freq, disable HW and buzzer code changes) still leaves the ACK not getting back reliably from the gateway.
I'll solder up ALL my motes and see what I can do with a completely systematic check all hardware with all combinations. Gives an excuse to break out excel :)
Oh and no delay was added - using the stock scripts with eh odd tweak for the specific hardware.
Alternatively, I should probably just wait for the actual mightyhat to arrive. All of this was temporary anyway to see if I couldn't get some node testing done.
Start from a set of hardware and software that works. Even if it means going back to basic examples.
You have to somehow narrow it down to either hardware or software.
That's the goal.
Of note, the MightyHat arrived today.
In case you wondering why you're missing a SonarMote label, you stuck it on a switchmote bag in the order. ;)
[edit]
BINGO!
Just finished construction and the gateway w/mightyhat and it's working perfectly with the node seeing immediate ACKs to information sent to the gateway. Yay!
Back to the initial question...
Is there a way to get data that is on the Raspberry pi into the logging system somehow? I can write the scripts and add the metrics, I'm just wondering what options I have with communication methods to the Gateway application that logs the node information? (either Software or Hardware)
Is the only way to send the data to an external node that then sends the data in the usual way via the FM module? I'm kinda hoping to avoid having to add another moteino on the gateway side just for this.
Soooo Awesome. Thanks again Felix!
I've managed to answer my own questions. I've eventually worked out that the MightyHat
is already listening on the Serial post for any instructions from the Pi. Github link to relevant line (https://github.com/LowPowerLab/RFM69/blob/master/Examples/MightyHat/MightyHat.ino#L320)
So armed with the knowledge, I understood that sending the correctly formatted text string from the Raspberry Pi was all that was needed. This would let me utilise my custom data generated by the Raspberry itself, and have it logged in the gateway software via the MightyHat.
UPDATE: DO NOT USE THIS CODE, it is causing 100%+ CPU usage. A node.js native solution for ISP speed monitoring will be provided in 8.12 release, please wait for that.
Step 1)
I added in my custom metrics in the file gateway/metrics.js
//ISP SpeedTest-CLI
PINGISP : { name:'PINGISP', regexp:/PINGISP\:([\d\.]+)/i, value:'', unit:'ms', pin:1, graph:1, graphOptions: { legendLbl:'Ping ms', lines: { lineWidth:1 }, colors:['#09c']} },
DOWNSPEED : { name:'DOWNSPEED', regexp:/DOWNSPEED\:([\d\.]+)/i, value:'', unit:'Mbs', pin:1, graph:1, graphOptions : { legendLbl:'Mb/s', lines: { lineWidth:1 }, colors:['#09c']} },
UPSPEED : { name:'UPSPEED', regexp:/UPSPEED\:([\d\.]+)/i, value:'', unit:'Mbs', pin:1, graph:1, graphOptions: { legendLbl:'Mb/sec', lines: { lineWidth:1 }, colors:['#09c']} },
Step 2)
I set up my PI to be an internet speed checker using these instructions:
https://thepi.io/how-to-use-your-raspberry-pi-to-monitor-broadband-speed/ (https://thepi.io/how-to-use-your-raspberry-pi-to-monitor-broadband-speed/)
Step 3)
I then modified the python script to send the node data to the MightyHat.
# required for sending data to Mighty Hat
import serial
# Might require installing pyserial
import os
import re
import subprocess
import time
response = subprocess.Popen('speedtest-cli --simple', shell=True, stdout=subprocess.PIPE).stdout.read()
ping = re.findall('Ping:\s(.*?)\s', response, re.MULTILINE)
download = re.findall('Download:\s(.*?)\s', response, re.MULTILINE)
upload = re.findall('Upload:\s(.*?)\s', response, re.MULTILINE)
ping[0] = ping[0].replace(',', '.')
download[0] = download[0].replace(',', '.')
upload[0] = upload[0].replace(',', '.')
try:
if os.stat('/home/pi/speedtest/speedtest.csv').st_size == 0:
print 'Date,Time,Ping (ms),Download (Mbit/s),Upload (Mbit/s)'
except:
pass
print '{},{},{},{},{}'.format(time.strftime('%m/%d/%y'), time.strftime('%H:%M'), ping[0], download[0], upload[0])
# code to send data to Mighty Hat
sp = serial.Serial("/dev/ttyAMA0", 19200, timeout=1.5)
sp.write("[99] PINGISP:" + str(ping[0]) + " DOWNSPEED:" + str(download[0]) + " UPSPEED:" + str(upload[0]) + "\n")
sp.close()
I haven't moved to fiber yet, so the attachment only shows my numbers for old ADSL+
Awesome, good work. I'm intrigued a little by the fact that you can open a secondary serial connection to the same port without any conflict.
I would have tried something like an event based nodejs function that polls your connection every X minutes, similar to what you did in python, but instead of going through the external serial port, this would keep it all contained in the gateway code. Anyway, if it already works, don't fix it :)
And as a tip:
Now use the labels of your metrics to include them in the node descriptions, that's something I use a lot in my UI to see my most relevant metrics right in the dashboard. So in your case:
Internet speed {DOWNSPEED}/{UPSPEED}
Or if you want to use the node's bubble then just PIN those metrics and they will show as space separated in the bubble. ;)
Awesome tip and thanks!
I need to rest up after getting that first node working. All this stuff is almost as bad as Factorio is in delaying sleep to make progress. :o
Next week: Weather/Water meter monitoring
Well, how about this:
Inspired by your work, and curious about my own internet speeds, I made my own version of this using native nodejs :D
I will release this along with some new features sometime soon. Stay tuned.
(https://i.imgur.com/7v82Wsw.png)
Ooooo stacked charts!
I was going to look into that. Great work as it looks fantastic to see it all in one view
well that is something I don't see, where I live. The biggest problem I ever had is then modem died. Was without internet for few hours. I don't have to deal with ping or speeds though, i'm limited by 100Mb/s lan connection of PI
OOO Felix!! stacked graphs!!! That is what many people don't even know that they want, can't wait for it!!
Something else to add to the Raspberry generated data I'll be adding:
vcgencmd measure_temp
temp=67.1'C
This will get the CPU Core temp from the Raspberry Pi and monitor that along with the internet speed. Anything else available for a standard RaspPI?
vcgencmd measure_temp
vcgencmd measure_clock arm
vcgencmd measure_volts core
free -o -h
some are constant I think
What I want to know is, why is you temp so high, my is 42°C pretty much all the time
I have no idea - For hardware, I have a mightyHat installed and a Lipo backup battery and a RasPi 3b+
For software, the Pi is running the usual gateway.js as the main process's that consuming > 60% of CPU all the time.
Is that a normal %?
I've reloaded both the MightyHatgateway.ino and the rasPi files.
On a full reboot of Pi, the CPU will start around 30% and ramp up to 106% within about 5 mins
MEM is about >10 to 40% for that process
Temp runs at a usual 60+
The 'top' command shows the primary command being 'node' - am I wrong to assume that is the gateway.js?
Also checked the pstree and it looks like there is some child process nodes (but I'm not sure I'm reading that screen right).
Not sure how else to troubleshoot this one. Anyone got additional ideas for things to check?
Hey,
no that is not normal. I have almost always 0%. In top (I don't know how you see each core) I have most of the time id >99.5%. The only difference is, I have pi3, not the last version, so here might be a problem with something that you have running.
When I run top, I have top process in the first spot, node come here and there
pstree give me 5 nodes and 4 workerthreads, so that is probably fine. I'm sure there is a problem using 3b+, just don't know how to find the problem
press '1' while in top to see the cores
I've tried both the latest release of gateway.js as well and the latest commit of gateway.js and both have the same results for CPU / temp.
I've 'sudo pkill node' a few times to kill the runaway process. The temp comes back for a bit, but a new node process starts up and gobbles up all the available ram and CPU and is back up there heating the pi at 60+ degree within 5mins. CPU% pegs out at 116%
huh, cool, thanks for that
and like i've said, my cpu is almost always idle. Do you have any older PI?
Nah - only dipped my toe into RasPi due to this Moteino project. I've only used Arduino Uno and Teensy otherwise.
I'm going to try running the following code from this post (https://stackoverflow.com/questions/41823465/how-to-debug-nodejs-causing-100-cpu-usage) to see what process is hogging the CPU
node --prof gateway.js
And run analysis with
node --prof-process generated_tick_file.log
Late Night EDIT:
*sigh* the title of this thread is about right - "Dummy" indeed...
Found that the opening of the serial port from the PI side to the MighyHat (in my case to upload the internet speed data) is what's causing the PI to 'go nuts' with CPU and memory.
Careful watching of the 'top' interface helped confirm the issue. I was able to run the gateway as above and then kicked off the internet download process. I watched the CPU spike immediately after.
So, that's being turned off as of right now :-[ and I'll hang on patiently for Felix to get around to looking how to do it his way.
I tried to edit the original post to recommend not to do it, but I can no longer edit the old post. I've reported it to the moderator to add a flag to not to use the suggested code.
I let the capture process run for 5mins in thrashing with high temp before terminating it. I've also attached the node analysis of the gateway.js in case Felix finds any useful information in it for future queries from dummies like me.
Just a thought - can I connect a USB cable from the Pi to the MightyHat and utilise the "/dev/ttyUSBx" rather that the previous "/dev/ttyAMA0" to avoid the conflict I created?
I'm assuming I'll need to snip the power cable in the USB cable to prevent the power looping from MightyHat to Raspberry pi back to the MightHat?
Yes, you can use either. I've tried both, connected Moteino to RPi, both worked the same. I would remove power yes, no need to open yourself up to potential problems, MH already has power, I would want to supply it over USB also.
Ergh, just realised the MightyHat doesn't have a seperate USB connector. I forgot that plug is for power.
Bugger it, I'll just wait for Felix and see what he has planned. At the rate I'm going, I'll probably blowupy RasPi or MH trying something.
Quote from: wile1411 on November 05, 2018, 11:55:46 PM
Just a thought - can I connect a USB cable from the Pi to the MightyHat and utilise the "/dev/ttyUSBx" rather that the previous "/dev/ttyAMA0" to avoid the conflict I created?
No, that's
only for power, you power the whole thing from the MHat's USB port, instead of using the Pi's USB input power port.
The serial is connected via GPIO directly. There are some jumpers on MHat which you can cut open and rewire it in some other way, but I've never done it so I don't have a specific tutorial on that for you.
Technically you could use an FTDIAdapter or similar USB-serial adapter to power it from the Pi's USB ports, and wire the FTDI header side to the MHat's serial port (only GND, TX/RX). Then you'd find a new serial port on the Pi emulated by the FTDIAdapter.
Thanks for the suggestion Felix. I might give it a go to see if I can get it working, even as just a proof of concept.
I'll hang tight and see what methods you have planned for getting the internet speeds recorded. Once that's out, I'll review the options and make a decision on if I need an additional dedicated FTDI adaptor for the 2nd serial connection option.
Sorry its taking so long.
I keep adding features, and I'm contemplating a major release (vs a minor). So likely a 9.0 at this point.
Wow - sorry my suggestion for comms between PI and HM prompted that amount work for you. :o Sound like you've dived a bit deep for these changes. I'm sure it's stuff you wanted anyway, but there's no hurry up from here. 8)
Happy to hang back and let you review / rewrite as often as you like. Looking at your release history for the gateway, you seem to do the occasional 6-8mth update that then gets followed be a few 1-2mth updates. Looks like you're in the later cycle at the moment. Hope you get back to the 6-8mth stable period soon :)
I've been piling up changes in the last year, a lot of which I checked in but I have not made into an official release. So to get the latest published changes, you can just run the install, which gets the latest available code in the repository (ie not a release) - not sure if I want to change this since most people prefer to be able to get latest beta features. I test everything for some time on my own before I publish any code anyway. A release is when I do a writeup about changes too, maybe a video - I hate documentation :)
Quote from: Felix on November 09, 2018, 09:25:49 AM
I hate documentation :)
Your not alone. But video makes things much better, when you can see new things in use
I appreciate all of your work....Thanks.
Video vs Documentation -- I am hard of hearing (wife quote: Deaf as a post) so videos really don't help me. Understand other points of view, so don't wish to add to work load. However, without the written word, I am pretty much out of the game.
Just two cents worth. Thanks again for your time!
I will try to document all major changes in written blog as well as a quick video that walks through what has changed. A lot of stuff is visual, so video makes most sense. For instance editing/deleting data from a graph, is very hard to illustrate in a static image.
Quote from: Felix on October 31, 2018, 10:21:34 PM
Well, how about this:
Inspired by your work, and curious about my own internet speeds, I made my own version of this using native nodejs :D
I will release this along with some new features sometime soon. Stay tuned.
(https://i.imgur.com/7v82Wsw.png)
Any update on this?
I am back at implementing changes, fixing some minor bugs and adding some features.
I hope I can release it "soon". I can't promis when, but this "soon" is sooner than the previous one for sure.
That is great to hear... yes, still can't wait, thanks!
Luka & those interested to experiment v9,
Just published V9 files.
Documentation is still pending but at least its ready to install.
Blog post with change log and feature showcase is here: https://lowpowerlab.com/2019/05/08/gateway-v9-released/
I would backup everything and install on a fresh latest raspbian image, then after everything works:
- systemctl stop gateway
- replace your data files from previous installation
- systemctl restart gateway
I'm on it! fire up pi3+ with USB drive instead of SD card.
I didn't do upgrade/ didn't work, but in <15min you can get it from scratch, if you move userMetric and db