Author Topic: Dummy Node for polling ISP down/up speeds  (Read 7601 times)

wile1411

  • Jr. Member
  • **
  • Posts: 54
  • Country: au
Dummy Node for polling ISP down/up speeds
« on: October 22, 2018, 12:24:35 AM »
Hi Felix & All,
I'm still in the process of implementing my PiGateway. I'm waiting on a MightyHat that it's currently flying it's way to Australia :)

While I'm waiting, I've been playing with the RaspberryPi 3B+ and getting things setup as far as I can. I've managed to successfully setup the Gateway software and have a no-ip account to access the Gateway from the internet and it all works well so far. So functionally it's all good, I just don't have a datasource to read yet.

What I'd like to do is load data into a dummy node. The data would be generated by the RasPi containing the internet upload / download speeds obtained by the speedtest-CLI package that I installed.

As I'm just getting started, I was wondering if someone could point me to the relevant area to learn about creating a dummy script to record the internet speed run by crontab for sending to the gateway listener process on the raspberry?

I (probably incorrectly) figure I should be able to do this while I don't have a mightyhat installed. Or am I out of luck until I actually connect something to the Serial port?


Changed my mind and decided to try and get a moteino connected to the gateway. After a bit of bumbling, I realized I forgot to do the step that would apply to a Pi3B+ where you requested to edit the /boot/config.txt with
Code: [Select]
enable_uart=1 
dtoverlay=pi3-disable-bt

As soon as I fixed that, everything mostly came up Milhouse! Although, I found that I'm getting a bit excessive NOK responses from the transmitter after it tries to do it's send. I did the check of the gateway and it's getting the message and sending the ACK but the sender doesn't seem to be able to reliably see the ACK.

I'm using both the standard PiGateway.ino script and the SonarMote_DistanceTracker.ino with only changes are commenting out the IS_RFM69HW_HCW and changing the frequency to 433Mhz. Would having the  SERIAL_EN enabled cause the transmitter to be too busy to respond to an ACK? I'm kind of grasping at straws as to why I'd see anything from 2 to 15 ACK:NOK before an ACK:OK

Now that I have a functional gateway, is there anything I can do to get the raspberry pi to communicate with its own serial port to send a dummy node packet to get the internet speed logging within the gateway interface?
« Last Edit: October 30, 2018, 02:53:10 PM by Felix »

Felix

  • Administrator
  • Hero Member
  • *****
  • Posts: 6866
  • Country: us
    • LowPowerLab
Re: Dummy Node for Gateway
« Reply #1 on: October 22, 2018, 10:53:01 AM »
If you have a W/CW then commenting out the #IS_RFM69HW_HCW setting is correct.
Matching settings correctly is crucial to getting a reliable link. Use default settings to start with, match frequency, network, and have separate IDs (1 for gateway), and the HW setting on each node.
Make sure antennas are well soldered, and they do not live in metal boxes.

Now that I have a functional gateway, is there anything I can do to get the raspberry pi to communicate with its own serial port to send a dummy node packet to get the internet speed logging within the gateway interface?
Not sure what you mean by "anything to get the internet speed logging within the gateway interface". How fast your internet is? No .. there is no code for that. Maybe you can write something up to poll an API or speedtest.net and report that in a dummy node somehow.

wile1411

  • Jr. Member
  • **
  • Posts: 54
  • Country: au
Re: Dummy Node for Gateway
« Reply #2 on: October 22, 2018, 05:55:35 PM »
Not sure what you mean by "anything to get the internet speed logging within the gateway interface". How fast your internet is? No .. there is no code for that. Maybe you can write something up to poll an API or speedtest.net and report that in a dummy node somehow.

I've got the code to do the speedtest process metrics, so that's no problem. As I understand it, it's easy enough to get an external moteino to send data to the gateway moteino which then passes it to the RasPi serial port to log using the Gateway software I installed.

What I'd like to do is have the raspberry itself be a data generator and send my speedtest data directly to the gateway software. I know I need to setup the metrics that I want to add into the metrics.js file and I think I can work that out based on the examples you've provided. Is it possible to have a py script (or something) connect to the listening gateway app and send the appropriately formatted data directly to the socket listener? Or is the only way to get data into the system is via the moteino that talks to the gateway?




If you have a W/CW then commenting out the #IS_RFM69HW_HCW setting is correct.
Matching settings correctly is crucial to getting a reliable link. Use default settings to start with, match frequency, network, and have separate IDs (1 for gateway), and the HW setting on each node.
Make sure antennas are well soldered, and they do not live in metal boxes.

There's no boxes as yet and yes I'm using the old FRM69W modules on all my moteinos. I've attached a few images to help explain my testing setup. I'm using the breadboard to connect the raspberry to the gateway moteino as I don't have any female-female cables and the test moteino already has the 90 degree header soldered on. Here is what's on the testing bench with the distance between the Gateway and the sonarmote about 80cm.

Transmitter SonarMote
Code: [Select]
// Sample RFM69 sender/node sketch for the SonarMote - Distance tracker
// Can be used for inventory control - ex to measure distance in a multi lane cigarette pack rack
// http://lowpowerlab.com/sonarmote
// Ultrasonic sensor (HC-SR04) connected to D6 (Trig), D7 (Echo), and power enabled through D5
// This sketch sleeps the Moteino and sensor most of the time. It wakes up every few seconds to take
//   a distance reading. If it detects an approaching object (car) it increases the sampling rate
//   and starts lighting up the LED (from green to yellow to red to blinking red). Once there is no more
//   motion the LED is turned off and the cycle is back to a few seconds in between sensor reads.
// Button is connected on D3. Holding the button for a few seconds enters the "red zone adjust" mode (RZA).
//   By default the red zone limit is at 25cm (LED turns RED below this and starts blinking faster and faster).
//   In RZA, readings are taken for 5 seconds. In this time you have the chance to set a new red zone limit.
//   Valid new red zone readings are between the RED__LIMIT_UPPER (default 25cm) and MAX_ADJUST_DISTANCE (cm).
//   In RZA mode the BLU Led blinks fast to indicate new red limit distance. It blinks slow if the readings are invalid
//   If desired this value could be saved to EEPROM to persist if unit is turned off
// Get the RFM69 at: https://github.com/LowPowerLab/
// Make sure you adjust the settings in the configuration section below !!!

// **********************************************************************************
// Copyright Felix Rusu 2016, http://www.LowPowerLab.com/contact
// **********************************************************************************
// License
// **********************************************************************************
// This program is free software; you can redistribute it
// and/or modify it under the terms of the GNU General   
// Public License as published by the Free Software       
// Foundation; either version 3 of the License, or       
// (at your option) any later version.                   
//                                                       
// This program is distributed in the hope that it will   
// be useful, but WITHOUT ANY WARRANTY; without even the 
// implied warranty of MERCHANTABILITY or FITNESS FOR A   
// PARTICULAR PURPOSE. See the GNU General Public       
// License for more details.                             
//                                                       
// Licence can be viewed at                               
// http://www.gnu.org/licenses/gpl-3.0.txt
//
// Please maintain this license information along with authorship
// and copyright notices in any redistribution of this code
// **********************************************************************************
#include <RFM69.h>    //get it here: https://www.github.com/lowpowerlab/rfm69
#include <SPI.h>      //included with Arduino IDE (www.arduino.cc)
#include <LowPower.h> //get library from: https://github.com/lowpowerlab/lowpower
                      //writeup here: http://www.rocketscream.com/blog/2011/07/04/lightweight-low-power-arduino-library/

//*********************************************************************************************
//************ IMPORTANT SETTINGS - YOU MUST CHANGE/CONFIGURE TO FIT YOUR HARDWARE *************
//*********************************************************************************************
#define NODEID        22   //unique for each node on same network
#define GATEWAYID     1    //node Id of the receiver we are sending data to
#define NETWORKID     200  //the same on all nodes that talk to each other including this node and the gateway
#define FREQUENCY     RF69_433MHZ //others: RF69_433MHZ, RF69_868MHZ (this must match the RFM69 freq you have on your Moteino)
// #define IS_RFM69HW_HCW  //uncomment only for RFM69HW/HCW! Leave out if you have RFM69W/CW!
#define ENCRYPTKEY    "sampleEncryptKey" //exactly the same 16 characters/bytes on all nodes!
#define SENDLOOPS    80 //default:80 //if no message was sent for this many sleep loops/cycles, then force a send
#define READ_SAMPLES 3
#define HISTERESIS   2  //(cm) only send a message when new reading is this many centimeters different
#define DIST_READ_LOOPS 2 //read distance every this many sleeping loops (ie if sleep time is 8s then 2 loops => a read occurs every 16s)

//*********************************************************************************************
//#define BUZZER_ENABLE  //uncomment this line if you have the BUZZER soldered and want the sketch to make sounds
#define SERIAL_EN      //uncomment if you want serial debugging output
//*********************************************************************************************
#define SLEEP_FASTEST SLEEP_15MS
#define SLEEP_FAST SLEEP_250MS
#define SLEEP_SEC SLEEP_1S
#define SLEEP_LONG SLEEP_2S
#define SLEEP_LONGER SLEEP_4S
#define SLEEP_LONGEST SLEEP_8S
period_t sleepTime = SLEEP_LONG; //period_t is an enum type defined in the LowPower library (LowPower.h)
//*********************************************************************************************
#ifdef __AVR_ATmega1284P__
  #define LED           15 // Moteino MEGAs have LEDs on D15
  #define FLASH_SS      23
#else
  #define LED           9 // Moteinos have LEDs on D9
  #define FLASH_SS      8
#endif
#define TRIG           6  // digital pin wired to TRIG pin of ultrasonic sensor
#define ECHO           7  // digital pin wired to ECHO pin of ultrasonic sensor
#define SENSOR_EN      5  // digital pin that enables power to ultrasonic sensor
#define BUZZER         4  // digital pin that is connected to onboard buzzer
#define MAX_DISTANCE 150  // maximum valid distance
#define MIN_DISTANCE   1.5  // minimum valid distance
#define MAX_ADJUST_DISTANCE (MAX_DISTANCE-GRN_LIMIT_UPPER)   //this is the amount by which the RED_LIMIT_UPPER can by increased

#ifdef SERIAL_EN
  #define SERIAL_BAUD   115200
  #define DEBUG(input)   {Serial.print(input);}
  #define DEBUGln(input) {Serial.println(input);}
  #define SERIALFLUSH() {Serial.flush();}
#else
  #define DEBUG(input);
  #define DEBUGln(input);
  #define SERIALFLUSH();
#endif

#define BATT_MONITOR  A7  // Sense VBAT_COND signal (when powered externally should read ~3.25v/3.3v (1000-1023), when external power is cutoff it should start reading around 2.85v/3.3v * 1023 ~= 883 (ratio given by 10k+4.7K divider from VBAT_COND = 1.47 multiplier)
#define BATT_READ_LOOPS  SENDLOOPS  // read and report battery voltage every this many sleep cycles (ex 30cycles * 8sec sleep = 240sec/4min). For 450 cycles you would get ~1 hour intervals between readings
#define BATT_FORMULA(reading) reading * 0.00322 * 1.475  // >>> fine tune this parameter to match your voltage when fully charged
#define BATT_LOW      3.55

byte sendLen;
byte sendLoops=0;
byte distReadLoops=0;
byte battReadLoops=0;
float distance=0;
float prevDistance=0;
float batteryVolts = 5;
char buff[50]; //this is just an empty string used as a buffer to place the payload for the radio
char* BATstr="BAT:5.00v"; //longest battery voltage reading message = 9chars
char* DISTstr="99999.99cm"; //longest distance reading message = 5chars
float readDistance(byte samples=1);  //take 1 samples by default
RFM69 radio;

void setup() {
#ifdef SERIAL_EN
  Serial.begin(SERIAL_BAUD); // Open serial monitor at 115200 baud to see ping results.
#endif

  radio.initialize(FREQUENCY,NODEID,NETWORKID);
#ifdef IS_RFM69HW_HCW
  radio.setHighPower(); //must include this only for RFM69HW/HCW!
#endif
  radio.encrypt(ENCRYPTKEY);
  //sprintf(buff, "\nTransmitting at %d Mhz...", FREQUENCY==RF69_433MHZ ? 433 : FREQUENCY==RF69_868MHZ ? 868 : 915);
  sprintf(buff, "\nTransmitting at %d Mhz, id:%d nid:%d gid:%d", FREQUENCY==RF69_433MHZ ? 433 : FREQUENCY==RF69_868MHZ ? 868 : 915, NODEID, NETWORKID, GATEWAYID);
  DEBUG(buff);
  for (byte i=0;i<strlen(ENCRYPTKEY);i++) DEBUG(ENCRYPTKEY[i]);
  DEBUGln();
  radio.sleep();

  pinMode(TRIG, OUTPUT);
  pinMode(ECHO, INPUT);
  pinMode(SENSOR_EN, OUTPUT);
  digitalWrite(SENSOR_EN, LOW);
#ifdef BUZZER_ENABLE
  pinMode(BUZZER, OUTPUT);
  buzzer(50,2,100);
#endif

  radio.sendWithRetry(GATEWAYID, "START", 5);

  SERIALFLUSH();
  readDistance(); //first reading seems to always be low
  readBattery();
}

void loop() {
  if (battReadLoops--<=0) //only read battery every BATT_READ_LOOPS cycles
  {
    readBattery();
    battReadLoops = BATT_READ_LOOPS-1;
  }
 
  if (distReadLoops--<=0)
  {
    distance = readDistance(READ_SAMPLES);
    distReadLoops = DIST_READ_LOOPS-1;
  }

  float diff = distance - prevDistance;
  if ((diff > HISTERESIS || diff < -HISTERESIS) || (sendLoops--<=0)) //only send a new message if the distance has changed more than the HISTERESIS or if sendloops has expired
  {
    if (distance > MAX_DISTANCE || distance < MIN_DISTANCE)
      DISTstr = "0"; // zero, out of range
    else dtostrf(distance,3,2, DISTstr);

    if (diff > HISTERESIS || diff < -HISTERESIS)
      sprintf(buff, "%scm BAT:%s", DISTstr, BATstr); //send both distance and battery readings
    else
      sprintf(buff, "BAT:%s", BATstr); //distance has not changed significantly so only send last battery reading
    sendLen = strlen(buff);

    digitalWrite(LED, HIGH);
    DEBUG(buff);
    if (radio.sendWithRetry(GATEWAYID, buff, sendLen))
    {
      prevDistance = distance;
      DEBUG(" - ACK:OK! RSSI:");
      DEBUGln(radio.RSSI);
    }
    else DEBUGln(" - ACK:NOK...");
    digitalWrite(LED, LOW);
    sendLoops = SENDLOOPS-1; //reset send loop counter
  }
  radio.sleep();
  SERIALFLUSH();

  LowPower.powerDown(sleepTime, ADC_OFF, BOD_OFF); //put microcontroller to sleep to save battery life
}

float readDistance(byte samples)
{
  if (samples == 0) samples = 1;
  if (samples > 10) samples = 10;
  digitalWrite(SENSOR_EN, HIGH);
  //need about 60-75ms after power up before HC-SR04 will be usable, so just sleep in the meantime
  LowPower.powerDown(SLEEP_60MS, ADC_OFF, BOD_OFF);
  LowPower.powerDown(SLEEP_15MS, ADC_OFF, BOD_OFF);
  PING();
  LowPower.powerDown(SLEEP_15MS, ADC_OFF, BOD_OFF);
 
  unsigned long duration = 0;
  for (byte i=0; i<samples; i++)
  {
    duration += PING();
    if (samples > 1) LowPower.powerDown(SLEEP_15MS, ADC_OFF, BOD_OFF);
  }
  digitalWrite(SENSOR_EN, LOW);
  return microsecondsToCentimeters(duration / samples);
}

long PING()
{
  digitalWrite(TRIG, LOW);
  delayMicroseconds(2);
  digitalWrite(TRIG, HIGH);
  delayMicroseconds(5);
  digitalWrite(TRIG, LOW);
  return pulseIn(ECHO, HIGH);
}

void readBattery()
{
  unsigned int readings=0;
  for (byte i=0; i<5; i++) //take several samples, and average
    readings+=analogRead(BATT_MONITOR);
  batteryVolts = BATT_FORMULA(readings / 5.0);
  dtostrf(batteryVolts,3,2, BATstr); //update the BATStr which gets sent every BATT_CYCLES or along with the MOTION message
  if (batteryVolts <= BATT_LOW) BATstr = "LOW";
}

float microsecondsToInches(long microseconds)
{
  // According to Parallax's datasheet for the PING))), there are
  // 73.746 microseconds per inch (i.e. sound travels at 1130 feet per
  // second).  This gives the distance travelled by the ping, outbound
  // and return, so we divide by 2 to get the distance of the obstacle.
  // See: http://www.parallax.com/dl/docs/prod/acc/28015-PING-v1.3.pdf
  return microseconds / 74.0 / 2.0f;
}

float microsecondsToCentimeters(long microseconds)
{
  // The speed of sound is 340 m/s or 29 microseconds per centimeter.
  // The ping travels out and back, so to find the distance of the
  // object we take half of the distance travelled.
  return (float)microseconds / 29.0f / 2.0f;
}

#ifdef BUZZER_ENABLE
void buzzer(byte soundTime, byte repeats, byte repeatsDelay)
{
  for (byte i=0;i<=repeats;i++)
  {
    tone(BUZZER, 4500); //4500hz makes a nice audible sound from a 3.3v Moteino digital pin
    delay(soundTime);
    noTone(BUZZER);
    if (repeats>0) delay(repeatsDelay);
  }
}
#endif

void Blink(byte pin)
{
  pinMode(pin, OUTPUT);
  digitalWrite(pin, HIGH);
  delay(2);
  digitalWrite(pin, LOW);
}



Gateway Moteino attached to RasPi
Code: [Select]
// **********************************************************************************************************
// Moteino gateway/base sketch that works with Moteinos equipped with RFM69W/RFM69HW/RFM69CW/RFM69HCW
// This is a basic gateway sketch that receives packets from end node Moteinos, formats them as ASCII strings
//      with the end node [ID] and passes them to Pi/host computer via serial port
//     (ex: "messageFromNode" from node 123 gets passed to serial as "[123] messageFromNode")
// It also listens to serial messages that should be sent to listening end nodes
//     (ex: "123:messageToNode" sends "messageToNode" to node 123)
// Make sure to adjust the settings to match your transceiver settings (frequency, HW etc).
// **********************************************************************************
// Copyright Felix Rusu 2016, http://www.LowPowerLab.com/contact
// **********************************************************************************
// License
// **********************************************************************************
// This program is free software; you can redistribute it
// and/or modify it under the terms of the GNU General   
// Public License as published by the Free Software       
// Foundation; either version 3 of the License, or       
// (at your option) any later version.                   
//                                                       
// This program is distributed in the hope that it will   
// be useful, but WITHOUT ANY WARRANTY; without even the 
// implied warranty of MERCHANTABILITY or FITNESS FOR A   
// PARTICULAR PURPOSE. See the GNU General Public       
// License for more details.                             
//                                                 
// Licence can be viewed at                               
// http://www.gnu.org/licenses/gpl-3.0.txt
//
// Please maintain this license information along with authorship
// and copyright notices in any redistribution of this code
// **********************************************************************************
#include <RFM69.h>         //get it here: https://github.com/lowpowerlab/rfm69
#include <RFM69_ATC.h>     //get it here: https://github.com/lowpowerlab/RFM69
#include <RFM69_OTA.h>     //get it here: https://github.com/lowpowerlab/RFM69
#include <SPIFlash.h>      //get it here: https://github.com/lowpowerlab/spiflash
#include <SPI.h>           //included with Arduino IDE (www.arduino.cc)

//****************************************************************************************************************
//**** IMPORTANT RADIO SETTINGS - YOU MUST CHANGE/CONFIGURE TO MATCH YOUR HARDWARE TRANSCEIVER CONFIGURATION! ****
//****************************************************************************************************************
#define NODEID          1 //the ID of this node
#define NETWORKID     200 //the network ID of all nodes this node listens/talks to
#define FREQUENCY     RF69_433MHZ //Match this with the version of your Moteino! (others: RF69_433MHZ, RF69_868MHZ)
#define ENCRYPTKEY    "sampleEncryptKey" //identical 16 characters/bytes on all nodes, not more not less!
// #define IS_RFM69HW_HCW  //uncomment only for RFM69HW/HCW! Leave out if you have RFM69W/CW!
#define ACK_TIME       50  // # of ms to wait for an ack packet
//*****************************************************************************************************************************
#define ENABLE_ATC    //comment out this line to disable AUTO TRANSMISSION CONTROL
#define ATC_RSSI      -75  //target RSSI for RFM69_ATC (recommended > -80)
//*****************************************************************************************************************************
// Serial baud rate must match your Pi/host computer serial port baud rate!
#define SERIAL_EN     //comment out if you don't want any serial verbose output
#define SERIAL_BAUD  19200
//*****************************************************************************************************************************

#ifdef __AVR_ATmega1284P__
  #define LED           15 // Moteino MEGAs have LEDs on D15
  #define FLASH_SS      23 // and FLASH SS on D23
#else
  #define LED           9 // Moteinos have LEDs on D9
  #define FLASH_SS      8 // and FLASH SS on D8
#endif

#ifdef SERIAL_EN
  #define DEBUG(input)   {Serial.print(input); delay(1);}
  #define DEBUGln(input) {Serial.println(input); delay(1);}
#else
  #define DEBUG(input);
  #define DEBUGln(input);
#endif

#ifdef ENABLE_ATC
  RFM69_ATC radio;
#else
  RFM69 radio;
#endif

SPIFlash flash(FLASH_SS, 0xEF30); //EF30 for 4mbit Windbond FlashMEM chip

void setup() {
  Serial.begin(SERIAL_BAUD);
  delay(10);
  radio.initialize(FREQUENCY,NODEID,NETWORKID);
#ifdef IS_RFM69HW_HCW
  radio.setHighPower(); //must include this only for RFM69HW/HCW!
#endif
  radio.encrypt(ENCRYPTKEY);
 
#ifdef ENABLE_ATC
  radio.enableAutoPower(ATC_RSSI);
#endif
 
  char buff[50];
  sprintf(buff, "\nTransmitting at %d Mhz...", radio.getFrequency()/1000000);
  DEBUGln(buff);

  if (flash.initialize())
  {
    DEBUGln("SPI Flash Init OK!");
  }
  else
  {
    DEBUGln("SPI FlashMEM not found (is chip onboard?)");
  }
}

byte ackCount=0;
byte inputLen=0;
char input[64];
byte buff[61];
String inputstr;

void loop() {
  inputLen = readSerialLine(input);
  inputstr = String(input);
  inputstr.toUpperCase();
 
  if (inputLen > 0)
  {
    if (inputstr.equals("KEY?"))
    {
      DEBUG("ENCRYPTKEY:");
      DEBUG(ENCRYPTKEY);
    }
   
    byte targetId = inputstr.toInt(); //extract ID if any
    byte colonIndex = inputstr.indexOf(":"); //find position of first colon

    if (targetId > 0)
    {
      inputstr = inputstr.substring(colonIndex+1); //trim "ID:" if any
    }

    if (targetId > 0 && targetId != NODEID && targetId != RF69_BROADCAST_ADDR && colonIndex>0 && colonIndex<4 && inputstr.length()>0)
    {
      inputstr.getBytes(buff, 61);
      //DEBUGln((char*)buff);
      //DEBUGln(targetId);
      //DEBUGln(colonIndex);
      if (radio.sendWithRetry(targetId, buff, inputstr.length()))
      {
        DEBUGln("ACK:OK");
      }
      else
        DEBUGln("ACK:NOK");
    }
  }

  if (radio.receiveDone())
  {
    int rssi = radio.RSSI;
    DEBUG('[');DEBUG(radio.SENDERID);DEBUG("] ");
    if (radio.DATALEN > 0)
    {
      for (byte i = 0; i < radio.DATALEN; i++)
        DEBUG((char)radio.DATA[i]);
      DEBUG("   [RSSI:");DEBUG(rssi);DEBUG("]");
    }

    CheckForWirelessHEX(radio, flash, false); //non verbose DEBUG

    if (radio.ACKRequested())
    {
      byte theNodeID = radio.SENDERID;
      radio.sendACK();
      DEBUG("[ACK-sent]");
    }
    DEBUGln();
    Blink(LED,3);
  }
}

void Blink(byte PIN, int DELAY_MS)
{
  pinMode(PIN, OUTPUT);
  digitalWrite(PIN,HIGH);
  delay(DELAY_MS);
  digitalWrite(PIN,LOW);
}

Actual TestBench showing moteinos. That's a deck of playing cards in front of the Sonarmote that I'll move a bit to see how the responses are.





Serial monitor of SonarMote showing lots of NOK which I can't work out why so many NOK. Do both sides need ATC? The test script only has the gateway with ATC so I tried adding ATC to both sides, but it didn't seem to help.



I tried to review the serial port with "sudo minicom -b 19200 -o -D /dev/ttyAMA0" and I can see that the gateway moteino get all the messages and I see the Gateway message saying "ACK sent".
The issue seems to be just the SonarMote moteino not seeing the ACK being sent from the gateway.

I'm pretty sure I must be doing something stupid or missing something obvious. I'm going to redo both moteinos and load the Struct_receive & Struct_send and see what results I get with those scripts on their own.
« Last Edit: October 23, 2018, 10:25:51 AM by wile1411 »

wile1411

  • Jr. Member
  • **
  • Posts: 54
  • Country: au
Re: Dummy Node for Gateway
« Reply #3 on: October 23, 2018, 04:46:49 AM »
Okie dokie... Struct_receive & Struct_send seems to work fine. See attached file output

I rechecked all my orders with the shop order history and have copy/pasted all transceivers here

Still to arrive:
14/10/2018 - R3 Mighty Hat (Transceiver: RFM69W - 433Mhz)
                  - R3 Motion Mote (Moteino type: Moteino + RFM69CW-433Mhz)

Have now:
27/09/2018 - R6 Moteino x1 (Transceiver: RFM69CW - 433Mhz)
12/04/2015 - R4 Moteino x4 (Transceiver: RFM69W - 433Mhz)
13/08/2014 - R4 Moteino x4 (Transceiver: RFM69W - 433Mhz)

These should all be 433Mhz according to my orders and should all work with each other as far as I've understood the moteino guide
For all this threads testing,  I was using a 2014 white spot for the gateway moteino and a 2015 Gold dot for the sonarmote moteino.
Switching to both white dots give the same result with a lot of NOKs but the Struct_receive & Struct_send seems to work good with AOKs for all messages.

Pictures below - (click for zoomable pic on imgur)
2014


2015


2018
« Last Edit: October 23, 2018, 10:35:50 AM by wile1411 »

Felix

  • Administrator
  • Hero Member
  • *****
  • Posts: 6866
  • Country: us
    • LowPowerLab
Re: Dummy Node for Gateway
« Reply #4 on: October 23, 2018, 03:56:28 PM »
First, there is a guide here showing all the types of transceivers, both with how they are marked, and how they look, this guide should help anyone to identify what they got:
https://lowpowerlab.com/guide/moteino/transceivers/

All yours seem to be the W/CW types, so your settings look correct. Sometimes close proximity can be an issue, I recall from other threads, and I think more at 433mhz but I could not be sure since I have not really observed this effect and I don't use 433mhz.

I would keep antennas straight up from the board, and away from other wiring, but this should not be a huge influence, as I understand your struct examples do work so that's good.
The PiGateway sketch automatically should send ACKs to sender motes.

Sanity check but can you confirm you don't happen to have another node with ID=22, or another gateway with ID=1 on the same network?

It would be easy enough to swap the Moteinos in question and see if anything changes, something you could check and report your findings?

wile1411

  • Jr. Member
  • **
  • Posts: 54
  • Country: au
Re: Dummy Node for Gateway
« Reply #5 on: October 23, 2018, 04:28:36 PM »
Thanks Felix,
Hahaha :) I can confirm I am sane. I've only got those two (1 and 22) that are even on the network or connected to anything. All 9 moteinos I own are in those pictures and I'm only using two of them.

proximity doesn't seem to be a thing as I still have good results with the structs. I did move the pi and gateway moteino across the room and no change in the results.

Could there be any timing issues I could troubleshoot? Also, is there any sanity checks I can run per board? I'm a bit lost what to do when the send/receive struct scripts work. So if that base level is fine, I'd expect it to be good for other scripts.

Lastly, there was one item I wanted to query in the struct scripts. What was the purpose of the radio.receiveDone() in the sending struct loop? Is there somewhere I can add that to verify the sonar script is not locking up in the sending area?
« Last Edit: October 23, 2018, 07:52:03 PM by wile1411 »

Felix

  • Administrator
  • Hero Member
  • *****
  • Posts: 6866
  • Country: us
    • LowPowerLab
Re: Dummy Node for Gateway
« Reply #6 on: October 24, 2018, 10:18:23 AM »
Timing issue? Not really, nothing I can think of right now.

The examples are given to have a clean start from something that works, they are used by others and I use them myself and there is no problem that I know of.
I haven't looked at all your code but if you add your own code to the sketch, and this is often overlooked, then I cannot vouch for that part, make sure you do not introduce any delays, I often see delay(xxxxxxxx) injected randomly (copy pasted from other sites for other sensors etc).

Why not try the Gateway and Node sketches. Those are barebones sketches and the node sends every 200ms, the gateway asks for an ACK every third packet.

wile1411

  • Jr. Member
  • **
  • Posts: 54
  • Country: au
Re: Dummy Node for Gateway
« Reply #7 on: October 24, 2018, 10:55:55 AM »
Thanks for the suggestions for the node and gateway.

Found one Moteino that as the gateway would fail to get the ACK back to the node. I'm going to reverse all the tests so each mote has a turn as both the gateway & node, but it looks like one
transceiver might be damaged to the point where it is shown getting "nothing..." a few times for those scripts.

Catch is, not matter which motes I check, all combinations once loaded with the PiGateway & Sonarmote scripts (with 433 freq, disable HW and buzzer code changes) still leaves the ACK not getting back reliably from the gateway.

I'll solder up ALL my motes and see what I can do with a completely systematic check all hardware with all combinations. Gives an excuse to break out excel :)

Oh and no delay was added - using the stock scripts with eh odd tweak for the specific hardware.

Alternatively, I should probably just wait for the actual mightyhat to arrive. All of this was temporary anyway to see if I couldn't get some node testing done.
« Last Edit: October 24, 2018, 11:00:49 AM by wile1411 »

Felix

  • Administrator
  • Hero Member
  • *****
  • Posts: 6866
  • Country: us
    • LowPowerLab
Re: Dummy Node for Gateway
« Reply #8 on: October 24, 2018, 11:03:01 AM »
Start from a set of hardware and software that works. Even if it means going back to basic examples.
You have to somehow narrow it down to either hardware or software.

wile1411

  • Jr. Member
  • **
  • Posts: 54
  • Country: au
Re: Dummy Node for Gateway
« Reply #9 on: October 25, 2018, 05:52:27 AM »
That's the goal.

Of note, the MightyHat arrived today.
In case you wondering why you're missing a SonarMote label, you stuck it on a switchmote bag in the order.  ;)

[edit]
BINGO!
Just finished construction and the gateway w/mightyhat and it's working perfectly with the node seeing immediate ACKs to information sent to the gateway. Yay!


Back to the initial question...
Is there a way to get data that is on the Raspberry pi into the logging system somehow? I can write the scripts and add the metrics, I'm just wondering what options I have with communication methods to the Gateway application that logs the node information? (either Software or Hardware)

Is the only way to send the data to an external node that then sends the data in the usual way via the FM module? I'm kinda hoping to avoid having to add another moteino on the gateway side just for this.
« Last Edit: October 25, 2018, 11:26:30 AM by wile1411 »

wile1411

  • Jr. Member
  • **
  • Posts: 54
  • Country: au
Re: Dummy Node for Gateway
« Reply #10 on: October 29, 2018, 07:56:52 AM »
Soooo Awesome. Thanks again Felix!

I've managed to answer my own questions. I've eventually worked out that the MightyHat
is already listening on the Serial post for any instructions from the Pi.  Github link to relevant line


So armed with the knowledge, I understood that sending the correctly formatted text string from the Raspberry Pi was all that was needed. This would let me utilise my custom data generated by the Raspberry itself, and have it logged in the gateway software via the MightyHat.

UPDATE: DO NOT USE THIS CODE, it is causing 100%+ CPU usage. A node.js native solution for ISP speed monitoring will be provided in 8.12 release, please wait for that.

Step 1)
I added in my custom metrics in the file gateway/metrics.js
Code: [Select]
  //ISP SpeedTest-CLI
  PINGISP : { name:'PINGISP', regexp:/PINGISP\:([\d\.]+)/i, value:'', unit:'ms', pin:1, graph:1, graphOptions: { legendLbl:'Ping ms', lines: { lineWidth:1 }, colors:['#09c']} },

  DOWNSPEED : { name:'DOWNSPEED', regexp:/DOWNSPEED\:([\d\.]+)/i, value:'', unit:'Mbs', pin:1, graph:1, graphOptions : { legendLbl:'Mb/s', lines: { lineWidth:1 }, colors:['#09c']} },

  UPSPEED : { name:'UPSPEED', regexp:/UPSPEED\:([\d\.]+)/i, value:'', unit:'Mbs', pin:1, graph:1, graphOptions: { legendLbl:'Mb/sec', lines: { lineWidth:1 }, colors:['#09c']} },

Step 2)
I set up my PI to be an internet speed checker using these instructions:
     https://thepi.io/how-to-use-your-raspberry-pi-to-monitor-broadband-speed/

Step 3)
I then modified the python script to send the node data to the MightyHat.
Code: [Select]
# required for sending data to Mighty Hat
import serial
# Might require installing pyserial

import os
import re
import subprocess
import time

response = subprocess.Popen('speedtest-cli --simple', shell=True, stdout=subprocess.PIPE).stdout.read()

ping = re.findall('Ping:\s(.*?)\s', response, re.MULTILINE)
download = re.findall('Download:\s(.*?)\s', response, re.MULTILINE)
upload = re.findall('Upload:\s(.*?)\s', response, re.MULTILINE)

ping[0] = ping[0].replace(',', '.')
download[0] = download[0].replace(',', '.')
upload[0] = upload[0].replace(',', '.')

try:
    if os.stat('/home/pi/speedtest/speedtest.csv').st_size == 0:
        print 'Date,Time,Ping (ms),Download (Mbit/s),Upload (Mbit/s)'
except:
    pass

print '{},{},{},{},{}'.format(time.strftime('%m/%d/%y'), time.strftime('%H:%M'), ping[0], download[0], upload[0])

# code to send data to Mighty Hat
sp = serial.Serial("/dev/ttyAMA0", 19200, timeout=1.5)
sp.write("[99] PINGISP:" + str(ping[0]) + " DOWNSPEED:" + str(download[0]) + " UPSPEED:" + str(upload[0]) + "\n")
sp.close()


I haven't moved to fiber yet, so the attachment only shows my numbers for old ADSL+
« Last Edit: November 05, 2018, 09:42:06 AM by Felix »

Felix

  • Administrator
  • Hero Member
  • *****
  • Posts: 6866
  • Country: us
    • LowPowerLab
Re: Dummy Node for Gateway
« Reply #11 on: October 29, 2018, 10:46:52 AM »
Awesome, good work. I'm intrigued a little by the fact that you can open a secondary serial connection to the same port without any conflict.
I would have tried something like an event based nodejs function that polls your connection every X minutes, similar to what you did in python, but instead of going through the external serial port, this would keep it all contained in the gateway code. Anyway, if it already works, don't fix it :)

And as a tip:
Now use the labels of your metrics to include them in the node descriptions, that's something I use a lot in my UI to see my most relevant metrics right in the dashboard. So in your case:

Internet speed {DOWNSPEED}/{UPSPEED}

Or if you want to use the node's bubble then just PIN those metrics and they will show as space separated in the bubble. ;)
« Last Edit: October 29, 2018, 01:37:38 PM by Felix »

wile1411

  • Jr. Member
  • **
  • Posts: 54
  • Country: au
Re: Dummy Node for Gateway
« Reply #12 on: October 29, 2018, 05:21:39 PM »
Awesome tip and thanks!
I need to rest up after getting that first node working. All this stuff is almost as bad as Factorio is in delaying sleep to make progress.  :o

Next week: Weather/Water meter monitoring

Felix

  • Administrator
  • Hero Member
  • *****
  • Posts: 6866
  • Country: us
    • LowPowerLab
Re: Dummy Node for polling ISP down/up speeds
« Reply #13 on: October 31, 2018, 10:21:34 PM »
Well, how about this:
Inspired by your work, and curious about my own internet speeds, I made my own version of this using native nodejs :D
I will release this along with some new features sometime soon. Stay tuned.


wile1411

  • Jr. Member
  • **
  • Posts: 54
  • Country: au
Re: Dummy Node for polling ISP down/up speeds
« Reply #14 on: October 31, 2018, 10:23:45 PM »
Ooooo stacked charts!
I was going to look into that. Great work as it looks fantastic to see it all in one view