Fresh Install of Pi Gateway - nginx problem

Started by Stereodude, June 20, 2024, 11:38:22 PM

brasskey4u

according to picocom, here is live data from my two weather nodes using /dev/serial0

pi@pigate:/dev $ sudo picocom -b 19200 /dev/serial0
picocom v3.1

port is        : /dev/serial0
flowcontrol    : none
baudrate is    : 19200
parity is      : none
databits are   : 8
stopbits are   : 1
escape is      : C-a
local echo is  : no
noinit is      : no
noreset is     : no
hangup is      : no
nolock is      : no
send_cmd is    : sz -vv
receive_cmd is : rz -vv -E
imap is        :
omap is        :
emap is        : crcrlf,delbs,
logfile is     : none
initstring     : none
exit_after is  : not set
exit is        : no

Type [C-a] [C-h] to see available commands
Terminal ready
[40] BAT:4.45v F:60.21 H:39.74 P:29.29   [RSSI:-48][ACK-sent]
[30] BAT:4.56v F:63.18 H:43.09 P:29.28   [RSSI:-63][ACK-sent]

So I am getting data from the MightyHat, but the gateway is still not running.  I did change the settings.json5 file to match my serial port that I am getting data from.  Web UI still has same error and gateway did not start.

brasskey4u

Also seeing this in gateway.sys.log   It keeps repeating the same error, perhaps that is another clue ?  The webserver.acccess.log and webserver.error.log are empty.

Node.js v25.6.0
/home/pi/gateway/gateway.js:1303



SyntaxError: Unexpected end of input
    at wrapSafe (node:internal/modules/cjs/loader:1740:18)
    at Module._compile (node:internal/modules/cjs/loader:1783:20)
    at Object..js (node:internal/modules/cjs/loader:1940:10)
    at Module.load (node:internal/modules/cjs/loader:1530:32)
    at Module._load (node:internal/modules/cjs/loader:1332:12)
    at wrapModuleLoad (node:internal/modules/cjs/loader:255:19)
    at Module.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:154:5)
    at node:internal/main/run_main_module:33:47

Felix

Probably related to not being able to start node.js or socket.io
I will try to do an install tomorrow if I have some time.

Felix

Ok I think I got it, it was a syntax error in one of the gateway.js commits.
File in HEAD is updated, can you replace your gateway.js with the latest in the repository, it should fix the problem and the gateway service should now start.
Otherwise install on a Pi5 with latest Pi OS Lite 64bit went flawlessly.

brasskey4u

Great news, well, kinda.  Got the webserver going and I selected two weather nodes, however there is no data, yet picocom shows data from both in the command line.  I will poke around in the nodes.js and see if something is off (no idea what I am looking for :)  ) 

Definitely closer than before and the gateway is running as you can see:

pi@pigate:~ $ ps ux | grep gateway
pi           966  0.0  0.1   2400  1460 ?        Ss   12:25   0:00 /bin/sh -c /usr/bin/node --trace-deprecation /home/pi/gateway/gateway.js >> /home/pi/gateway/logs/gateway.sys.log 2>&1
pi           976  0.2  7.8 1391224 72664 ?       Sl   12:25   0:07 /usr/bin/node --trace-deprecation /home/pi/gateway/gateway.js
pi          1230  0.0  0.2   6252  2008 pts/0    S+   13:11   0:00 grep --color=auto gateway
pi@pigate:~ $

brasskey4u

I do see this and I have checked that my serial port is set to serial0, unclear as to where to go to check this error.
pi@pigate:~/gateway/logs $ tail -f gateway.sys.log
[26/02/10 13:20:39.943] [INFO]  settings.serial.enabled FALSE or NOT FOUND!
[26/02/10 13:20:40.945] [INFO]  settings.serial.enabled FALSE or NOT FOUND!
[26/02/10 13:20:41.946] [INFO]  settings.serial.enabled FALSE or NOT FOUND!
[26/02/10 13:20:42.947] [INFO]  settings.serial.enabled FALSE or NOT FOUND!
[26/02/10 13:20:43.947] [INFO]  settings.serial.enabled FALSE or NOT FOUND!
[26/02/10 13:20:44.949] [INFO]  settings.serial.enabled FALSE or NOT FOUND!
[26/02/10 13:20:45.950] [INFO]  settings.serial.enabled FALSE or NOT FOUND!
[26/02/10 13:20:46.951] [INFO]  settings.serial.enabled FALSE or NOT FOUND!
[26/02/10 13:20:47.952] [INFO]  settings.serial.enabled FALSE or NOT FOUND!
[26/02/10 13:20:48.953] [INFO]  settings.serial.enabled FALSE or NOT FOUND!
[26/02/10 13:20:49.954] [INFO]  settings.serial.enabled FALSE or NOT FOUND!
[26/02/10 13:20:50.955] [INFO]  settings.serial.enabled FALSE or NOT FOUND!
[26/02/10 13:20:51.956] [INFO]  settings.serial.enabled FALSE or NOT FOUND!
[26/02/10 13:20:52.957] [INFO]  settings.serial.enabled FALSE or NOT FOUND!
[26/02/10 13:20:53.957] [INFO]  settings.serial.enabled FALSE or NOT FOUND!
[26/02/10 13:20:54.958] [INFO]  settings.serial.enabled FALSE or NOT FOUND!
[26/02/10 13:20:55.958] [INFO]  settings.serial.enabled FALSE or NOT FOUND!
[26/02/10 13:20:56.960] [INFO]  settings.serial.enabled FALSE or NOT FOUND!
[26/02/10 13:20:57.961] [INFO]  settings.serial.enabled FALSE or NOT FOUND!


Felix

Found another issue, I forgot to merge a new serial.enabled setting, had it locally.
Get the latest settings.json5 and restart the gateway (CTRL+SHIFT+ALT+R).

brasskey4u

That was it!  I got my 2 weather sensors showing good data, but still nothing from the Sonar Mote, yet I get data from it via picocom and the App server disconnects and then starts and again, good weather nodes, just nothing from the basement mote.  Perhaps I need to upload a new sketch to it.  I did find a bad header that the ultrasonic  module plugs into, one of the pins was broken internally, so I put a new header in and get distance now.  Earlier was just getting signal strength and battery voltage.  Thanks a million Felix, I really appreciate all the time you put into this.  Nice to have this working again.  I moved and when I fired up the gateway it didn't work.  I really like seeing the temp data graphed over time.  Even put one in my attic at my old house to see how hot it got in the summer.  I will try a new sketch on the mote unless you think something might be off.  I haven't done anything with the 3 nodes since I bought them from you years ago, I just followed the instructions and loaded the sketches and off they went.  While I am writing this I have seen the web page refresh several times now.


brasskey4u

OK, got the Sonar Mote working now too.  The battery was bad (kind of thought so because it felt slightly warm).  I put a new battery in and I am getting data on the Web UI too,  now If I can figure out why the web page keeps refreshing.  Seems like anytime there is new data from a node it restarts.  I get the message:

App server was disconnected
Waiting for new connection.

Then it comes back up to normal.

This from the terminal window:
6:05:38 PM : Disconnected!
18:05:38.610 : /dev/serial0>:[30] BAT:4.55v F:61.03 H:49.51 P:29.15   [RSSI:-63][ACK-sent]
18:03:53.260 : SERVERTIME OFFSET: -783ms
18:03:53.224 : Connected!
6:03:34 PM : Disconnected!
18:03:34.214 : /dev/serial0>:[30] BAT:4.55v F:61.02 H:49.53 P:29.15   [RSSI:-63][ACK-sent]
18:01:42.253 : SERVERTIME OFFSET: -793ms
18:01:42.205 : Connected!
6:01:27 PM : Disconnected!
18:01:27.827 : /dev/serial0>:[40] BAT:4.45v F:67.89 H:34.25 P:29.14   [RSSI:-66][ACK-sent]
17:59:30.575 : SERVERTIME OFFSET: -743ms
17:59:30.522 : Connected!
5:59:14 PM : Disconnected!
17:59:14.749 : /dev/serial0>:[40] BAT:4.45v F:67.71 H:34.43 P:29.14   [RSSI:-66][ACK-sent]
17:58:02.645 : SERVERTIME OFFSET: -766ms
17:58:02.586 : Connected!

Felix

Probably because the gateway.js app restarts. See if there's something more specific in gateway.sys.log

brasskey4u

I see this over and over in the log:

pi@pigate:~/gateway/logs $ tail -100 gateway.sys.log
    at /home/pi/gateway/node_modules/nedb/lib/persistence.js:201:12
    at FSReqCallback.oncomplete (node:fs:180:23)

Node.js v25.6.1
[26/02/11 10:45:12.308] [INFO]  *********************************************************************
[26/02/11 10:45:12.314] [INFO]  ************************* GATEWAY APP START *************************
[26/02/11 10:45:12.315] [INFO]  *********************************************************************
[26/02/11 10:45:12.358] [INFO]  LOADING METRICS MODULE [./metrics/core.js]
[26/02/11 10:45:12.362] [INFO]  |- GLOBALIZING ONEDAY
[26/02/11 10:45:12.364] [INFO]  |- GLOBALIZING isNumeric()
[26/02/11 10:45:12.365] [INFO]  |- GLOBALIZING isValidIP()
[26/02/11 10:45:12.366] [INFO]  |- GLOBALIZING isValidNodeId()
[26/02/11 10:45:12.367] [INFO]  |- GLOBALIZING determineValue()
[26/02/11 10:45:12.368] [INFO]  |- GLOBALIZING determineGraphValue()
[26/02/11 10:45:12.370] [INFO]  |- GLOBALIZING timeoutOffset()
[26/02/11 10:45:12.371] [INFO]  |- GLOBALIZING millisToFutureDate()
[26/02/11 10:45:12.372] [INFO]  |- GLOBALIZING nextSunriseOrSunset()
[26/02/11 10:45:12.452] [INFO]  LOADING METRICS MODULE [/home/pi/gateway/metrics/_LowPowerLab/doorbellmote.js]
[26/02/11 10:45:12.456] [INFO]  LOADING METRICS MODULE [/home/pi/gateway/metrics/_LowPowerLab/garagemote.js]
[26/02/11 10:45:12.470] [INFO]  LOADING METRICS MODULE [/home/pi/gateway/metrics/_LowPowerLab/motionmote.js]
[26/02/11 10:45:12.474] [INFO]  LOADING METRICS MODULE [/home/pi/gateway/metrics/_LowPowerLab/RadioThermostat_CT50.js]
[26/02/11 10:45:12.480] [INFO]  LOADING METRICS MODULE [/home/pi/gateway/metrics/_LowPowerLab/sonarmote.js]
[26/02/11 10:45:12.483] [INFO]  LOADING METRICS MODULE [/home/pi/gateway/metrics/_LowPowerLab/sprinklers.js]
[26/02/11 10:45:12.488] [INFO]  LOADING METRICS MODULE [/home/pi/gateway/metrics/_LowPowerLab/switchmote.js]
[26/02/11 10:45:12.492] [INFO]  LOADING METRICS MODULE [/home/pi/gateway/metrics/_LowPowerLab/watermeter.js]
[26/02/11 10:45:12.495] [INFO]  LOADING METRICS MODULE [/home/pi/gateway/metrics/_LowPowerLab/weathershield.js]
[26/02/11 10:45:12.499] [INFO]  LOADING METRICS MODULE [/home/pi/gateway/metrics/examples/_example.js]
[26/02/11 10:45:12.504] [INFO]  |- GLOBALIZING ONEDAYHOURS
[26/02/11 10:45:12.505] [INFO]  |- GLOBALIZING secondsInOneDay()
[26/02/11 10:45:12.506] [INFO]  LOADING METRICS MODULE [/home/pi/gateway/metrics/examples/_garage-auto-close.js]
[26/02/11 10:45:12.509] [INFO]  LOADING METRICS MODULE [/home/pi/gateway/metrics/examples/_InternetSpeedTest.js]
[26/02/11 10:45:12.558] [INFO]  SERIAL PORT CONNECTION TO: /dev/serial0 @ 19200baud
[26/02/11 10:45:12.695] [INFO]  Available serial port: {path:'/dev/ttyS0'}
[26/02/11 10:45:12.709] [LOG]   /dev/serial0>:GTWCMD:UPTIME
[26/02/11 10:45:12.713] [LOG]   /dev/serial0>:GTWCMD:FREERAM
[26/02/11 10:45:12.715] [LOG]   /dev/serial0>:GTWCMD:SYSFREQ
[26/02/11 10:45:12.716] [LOG]   /dev/serial0>:GTWCMD:UPTIME
[26/02/11 10:45:12.717] [LOG]   /dev/serial0>:GTWCMD:FREERAM
[26/02/11 10:45:12.719] [LOG]   /dev/serial0>:GTWCMD:SYSFREQ
[26/02/11 10:45:12.720] [LOG]   /dev/serial0>:GTWCMD:GATEWAYFWVERSION
[26/02/11 10:45:12.721] [LOG]   /dev/serial0>:GTWCMD:GATEWAYFWNAME
[26/02/11 10:45:12.723] [LOG]   /dev/serial0>:GTWCMD:FLXIMG
[26/02/11 10:45:12.725] [LOG]   /dev/serial0>:GTWCMD:NETID
[26/02/11 10:45:15.440] [LOG]   /dev/serial0>:[22] .93   [RSSI:-68][ACK-sent]
[26/02/11 10:45:15.463] [LOG]   NODEDATA PARTIAL NOMATCH>: .93
[26/02/11 10:45:15.468] [LOG]   post: /home/pi/gateway/data/db/0022_RSSI.bin[1770824715,-68]
[26/02/11 10:45:15.472] [LOG]   NODEDATA PARTIAL NOMATCH>: [RSSI:-68][ACK-sent]
/home/pi/gateway/gateway.js:981
          console.dbupdate(`[${id}] updated`);
                  ^

TypeError: console.dbupdate is not a function
    at /home/pi/gateway/gateway.js:981:19
    at newArguments.<computed> (/home/pi/gateway/node_modules/nedb/lib/executor.js:29:17)
    at /home/pi/gateway/node_modules/nedb/lib/datastore.js:643:18
    at /home/pi/gateway/node_modules/nedb/lib/persistence.js:201:12
    at FSReqCallback.oncomplete (node:fs:180:23)


Felix

That's weird, that should produce a log entry like [DBUPDATE] instead of [LOG]. Essentially it could be anything.
In any case this works on my end. But to get past this you could deplace
console.dbupdate
with
console.log

brasskey4u

OK, that was it!  No more restarts.  So hopefully my last question.  What is the EASY way to back up this entire SD card or make a copy of it?  I would like to have a full back up of this so I don't have all this "fun" next time.  If you know of any utilities I would be most thankful.  Appreciate the time you took with me on this.  Hopefully our conversations will help someone else.  By the way, I did not change anything on the MightyHat or the 2 weather nodes or sonar mote, just wanted to throw that out there.  Only the battery in the Mote was replaced.  Man is this nice to have this all back.  Have a great day.  Thanks again.

Felix

That's somewhere in my mile long TODO list for the PiGateway project. A script or some way to back/zip up everything in a single archive that's compact and easy to unpack and replace everything later. But it's probably more involved than it sounds because of dependencies in node.js.

For now the way I do it is just pop out the SD and image it. I use industrial grade SD's though so I have not seen a single SD corruption fail in forever. There is minimal wear because the logs are cyclical and the database is incremental. The OS is probably where most "wear" is. But you can always use an SSD now with Pi+NVME/M.2 hats.

brasskey4u

Yes, I think just copying the SD card will be the best bet for now.  I'm inspired now to try something else with the other Motenio's I have and get a few more nodes fired up.  Do you have any plans to make some of the back ordered projects again?  I'm guessing you have other irons in the fire now.  So many IoT devices to play with now than years ago.  I have Home Assistant, but have yet to really dive into it.  Seems somewhat clumsy and not very intuitive for my old brain.