Gateway v9.0.0 upgrade issue [solved: clean install]

[Lukapple]

Hi,
I've upgraded to v9 (didn't do the clean install of raspbian, because I have other things installed on this pi, like VPN and HomeBridge) and now when I try to login (web interface), it gets stuck on screen "connected... waiting for data".
If I check the gateway logs, everything looks ok - server is getting data from nodes:

Code Select Expand

...
[18.06.19_08:25:21.451] [LOG]    >: [4] BAT:2.71v   [RSSI:-75][ACK-sent]
[18.06.19_08:25:21.454] [LOG]    post: /home/pi/gateway/data/db/0004_V.bin[1560839121,2.71]
[18.06.19_08:25:21.455] [LOG]    post: /home/pi/gateway/data/db/0004_RSSI.bin[1560839121,-75]
[18.06.19_08:25:21.463] [LOG]       [4] DB-Updates:1
...

I've upgraded to v9 this way:
1. stopped gateway and renamed gateway to gateway_OLD
2. I wanted to keep current version of node (v 7.10.1), because of some HomeBridge dependencies, so  used modified script:
https://raw.githubusercontent.com/LowPowerLab/RaspberryPi-Gateway/master/.setup/Install-Node.sh
I commented out Node install part:
#sudo wget -O - https://raw.githubusercontent.com/LowPowerLab/RaspberryPi-Gateway/master/.setup/Install-Node.sh | sudo bash

Also checked NginX logs and there I've noticed HTTP 304 (not modified) code - is that normal?

Code Select Expand

192.168.x.x - luka [18/Jun/2019:08:48:43 +0200] "GET /socket.io/socket.io.js HTTP/1.1" 304 0 "https://ip_removed/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.1.1 Safari/605.1.15"
192.168.x.x - luka [18/Jun/2019:08:48:43 +0200] "GET /socket.io/?EIO=3&transport=polling&t=MjfM53E HTTP/1.1" 200 99 "https://ip_removed/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.1.1 Safari/605.1.15"
192.168.x.x - luka [18/Jun/2019:08:48:44 +0200] "GET /socket.io/?EIO=3&transport=polling&t=MjfM53Z&sid=HaPIm_2rI5gTxbpkAAAL HTTP/1.1" 200 6438 "https://ip_removed/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.1.1 Safari/605.1.15"
192.168.x.x - - [18/Jun/2019:08:48:44 +0200] "GET /socket.io/?EIO=3&transport=websocket&sid=HaPIm_2rI5gTxbpkAAAL HTTP/1.1" 401 195 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.1.1 Safari/605.1.15"

Any idea why web interface doesn't work? 

[Kylix]

It would be useful to check gateway log file.

[Lukapple]

Already did, and nothing unusual there.
Here is log after gateway restart:

gateway/logs/gateway.sys.log:

Code Select Expand

[18.06.19_10:27:48.592] [INFO]   LOADING USER METRICS...
[18.06.19_10:27:48.606] [INFO]   LOADING USER METRICS MODULE [_InternetSpeedTest.js]
[18.06.19_10:27:48.638] [INFO]   LOADING USER METRICS MODULE [_example.js]
[18.06.19_10:27:48.653] [INFO]   LOADING USER METRICS MODULE [metrics_genisys.js]
[18.06.19_10:27:48.689] [LOG]    Express server listening on port 3000
[18.06.19_10:27:48.792] [INFO]   AUTHORIZING CONNECTION FROM ::ffff:127.0.0.1:33876
[18.06.19_10:27:48.816] [INFO]   NEW CONNECTION FROM ::ffff:127.0.0.1
[18.06.19_10:27:48.851] [INFO]   AUTHORIZING CONNECTION FROM ::ffff:127.0.0.1:33878
[18.06.19_10:27:48.856] [INFO]   NEW CONNECTION FROM ::ffff:127.0.0.1
[18.06.19_10:27:49.025] [INFO]   AUTHORIZING CONNECTION FROM ::ffff:127.0.0.1:33888
[18.06.19_10:27:49.028] [INFO]   NEW CONNECTION FROM ::ffff:127.0.0.1
[18.06.19_10:27:56.957] [INFO]   AUTHORIZING CONNECTION FROM ::ffff:127.0.0.1:33898
[18.06.19_10:27:56.959] [INFO]   NEW CONNECTION FROM ::ffff:127.0.0.1
[18.06.19_10:27:56.974] [INFO]   AUTHORIZING CONNECTION FROM ::ffff:127.0.0.1:33900
[18.06.19_10:27:56.976] [INFO]   NEW CONNECTION FROM ::ffff:127.0.0.1
[18.06.19_10:27:56.986] [INFO]   AUTHORIZING CONNECTION FROM ::ffff:127.0.0.1:33902
[18.06.19_10:27:56.987] [INFO]   NEW CONNECTION FROM ::ffff:127.0.0.1
[18.06.19_10:27:57.000] [INFO]   AUTHORIZING CONNECTION FROM ::ffff:127.0.0.1:33904
[18.06.19_10:27:57.001] [INFO]   NEW CONNECTION FROM ::ffff:127.0.0.1
[18.06.19_10:27:57.566] [LOG]    **** DB COMPACTION DONE
[18.06.19_10:27:57.568] [LOG]    **** DB COMPACTION DONE
[18.06.19_10:27:57.569] [LOG]    **** DB COMPACTION DONE
[18.06.19_10:27:57.570] [LOG]    **** DB COMPACTION DONE
[18.06.19_10:27:57.572] [LOG]    **** DB COMPACTION DONE
[18.06.19_10:27:57.573] [LOG]    **** DB COMPACTION DONE
[18.06.19_10:27:57.574] [LOG]    **** DB COMPACTION DONE
[18.06.19_10:27:57.595] [LOG]    **** SCHEDULING EVENT - nodeId:11 event:sendTimeSyncDMD to run in ~16h32m
[18.06.19_10:27:57.702] [LOG]       [11] DB-Updates:1
[18.06.19_10:28:01.375] [LOG]    >: [7] 104.5cm   [RSSI:-58][ACK-sent]
[18.06.19_10:28:01.407] [LOG]    post: /home/pi/gateway/data/db/0007_Količina.bin[1560846481,395.91]
[18.06.19_10:28:01.420] [LOG]    post: /home/pi/gateway/data/db/0007_RSSI.bin[1560846481,-58]
[18.06.19_10:28:01.434] [LOG]       [7] DB-Updates:1
[18.06.19_10:28:07.494] [INFO]   AUTHORIZING CONNECTION FROM ::1:33576
[18.06.19_10:28:07.495] [INFO]   NEW CONNECTION FROM 192.168.2.38

Those new connections from 127.0.0.1 are connections from HomeBridge plugin, and at the end (.38) is connection from web interface.

[Felix]

Not sure why you'd get a HTTP 304, but its just a not modified, I guess nginx can send that status to the browser if a request is the same and not modified.
I think this is not related to what you see though.

Can you try a clean install on a new SD card? That should work.
Any possibility you upgrade your other stuff to the newer versions of the packages?
I strongly recommend upgrading to the latest nodeJS in the setup.

[Lukapple]

Looks like I'll have to do a clean install. I'll let you know if this will solve the issue. Thanks.

[Felix]

For now yes, the version to upgrade to is hardcoded in the script.
Every time I upgrade I have to verify the latest available version can actually install and run without issues on the Pi.
Hence the discrepancy.

[Lukapple]

Ok, now it works perfect with clean install 
Now I have to try some custom HTTP nodes 
Thanks Felix for fantastic work!

[Felix]

I haven't used the HTTP endpoint too extensively yet.
Also I found other quirks which I already fixed and testing, and will probably make a new release soon.

[Lukapple]

What I want to do is to add a dummy node, which will call a js script, that returns a temperature from Netatmo weather station (they have API and I've already prepared that js script).
Then I'll add gateway event, which will forward temperature to my DMD(Dot Matrix Display) node, where I can already see a water temperature from my pool 

[Felix]

Pretty cool, so this HTTP/endpoint functionality can help you achieve that?

[Lukapple]

I think that this is exactly what I need, but I didn't have time to test it yet. I'll try tonight and let you know if it works.

[Lukapple]

Just to let you know, polling data from Netatmo weather station works perfect with both methods:
- HTTP/endpoint
and
- metrics events (same way as speedtest.net event)

I've chosed to use metrics method - didn't know that it's so simple to use.

Thanks again for this fantastic v9 gateway version!