LowPowerLab Forum

Software support => Pi Gateway => Topic started by: TrickyT on September 23, 2022, 05:39:20 PM

Title: Upgrading NodeJS causes gateway.js errors
Post by: TrickyT on September 23, 2022, 05:39:20 PM
I upgraded Node to v18.9.0 because another application I run required this version. But now gateway gets the following error:

/home/tef/gateway/gateway.js:834
          if (metricsDef.metrics[metric].regexp.test(match[0]))

TypeError: metricsDef.metrics[metric].regexp.test is not a function
    at /home/tef/gateway/gateway.js:834:49
    at newArguments.<computed> (/home/tef/gateway/node_modules/nedb/lib/executor.js:29:17)
    at Cursor.execFn (/home/tef/gateway/node_modules/nedb/lib/datastore.js:518:14)
    at callback (/home/tef/gateway/node_modules/nedb/lib/cursor.js:126:19)
    at /home/tef/gateway/node_modules/nedb/lib/cursor.js:193:12
    at /home/tef/gateway/node_modules/nedb/lib/datastore.js:329:14
    at async.eachSeries (/home/tef/gateway/node_modules/nedb/node_modules/async/lib/async.js:130:20)
    at /home/tef/gateway/node_modules/nedb/lib/datastore.js:323:11
    at fn (/home/tef/gateway/node_modules/nedb/node_modules/async/lib/async.js:582:34)
    at Immediate.<anonymous> (/home/tef/gateway/node_modules/nedb/node_modules/async/lib/async.js:498:34)
Node.js v18.9.0


I'm a javascript noob, but I did google around a bit and it seems like metricsDef.metrics[metric] needs to be a regular expression.
There are several places in gateway.js where syntax like this is used, but in one place where the if() is failing metricsDef.metrics[metric] is set to "{ name: 'DEBUG', regexp: {}, value: ' ' }" and match[0] equals the string "F:99.21". Can someone please suggest changes to gateway.js to fix this problem.

Thanks!

(EDIT: fix typo)
Title: Re: Upgrading NodeJS causes gateway.js errors
Post by: Felix on September 26, 2022, 07:44:19 AM
Hi,
Did you just update node? How about node js and reinstall the packages used by the gateway app?
There is a guide here for that:
https://lowpowerlab.com/guide/gateway/node-npm-updates/
Let me know if this helps.
Title: Re: Upgrading NodeJS causes gateway.js errors
Post by: TrickyT on September 27, 2022, 02:50:20 PM
I followed the NodeJS & npm update guide. That seemed to go just fine and created a new directory of node_modules. Unfortunately gateway.js encounters exactly the same error as I described previously.
Title: Re: Upgrading NodeJS causes gateway.js errors
Post by: Felix on September 27, 2022, 05:24:04 PM
There is nothing wrong with gateway.js.
It is the metric that is not correct, as you can see in the core.js metrics file (https://github.com/LowPowerLab/RaspberryPi-Gateway/blob/master/metrics/core.js), the DEBUG metric does have a valid regex. Please check your metrics files against this.
Title: Re: Upgrading NodeJS causes gateway.js errors
Post by: TrickyT on September 27, 2022, 08:13:41 PM
OK, will do. Thanks for the tip. I haven't modified any of my metrics files in well over a year, so perhaps the new version of node js uses a more strict interpretation of some language feature. Will report back after I figure it out.
Title: Re: Upgrading NodeJS causes gateway.js errors
Post by: TrickyT on September 28, 2022, 07:52:27 PM
The metrics definitions are all fine. The DEBUG metric does have a valid regexp defined in the core.js module, but after this module is loaded and merge() is called, javascript no longer knows it's a regexp. If I call "console.log(metricsDef.metrics[metric].regexp.toString());" for every metric in core.js and _LowPowerLab/*.js and my local metric definitions, what gets logged to the console every time is "[object Object]" while all the other properties like "name" and "value" display what is expected. What should be getting logged is a string containing the regular expression. The runtime error getting logged when gateway tries to find a matching metric...

    TypeError: metricsDef.metrics.regexp.test is not a function

is because javascript doesn't know the object is a regexp and doesn't think the object has a test() method.
Title: Re: Upgrading NodeJS causes gateway.js errors
Post by: TrickyT on September 28, 2022, 08:19:01 PM
Here's a test program to illustrate the problem:

var merge = require('merge');
var tmp = require('./metrics/core.js');

global.metricsDef = {};
metricsDef.metrics = merge(true, metricsDef.metrics, tmp.metrics);
for(var metric in metricsDef.metrics)
{
  console.log('metric: ' + metricsDef.metrics[metric].name);
  console.log(metricsDef.metrics[metric].regexp.toString());
}


...which produces the following...

metric: DEBUG
[object Object]
metric: V
[object Object]
metric: RSSI
[object Object]
metric: TXLVL
[object Object]
metric: START
[object Object]
metric: TYPE
[object Object]
metric: P_TXP
[object Object]


EDIT: I know the metric definitions in core.js are read in correctly. It's the merge() that's failing.
Title: Re: Upgrading NodeJS causes gateway.js errors
Post by: TrickyT on September 30, 2022, 12:20:05 PM
OK, my issue has been resolved.

When I used npm to re-install all the javascript packages, it chose a newer implementation of the merge module. Newer (broken) version 2.1.1, older (working) version 1.2.1. Both come from https://github.com/yeikos/js.merge (https://github.com/yeikos/js.merge). I re-installed the older version from a backup and gateway is working again.

This is a known issue in version 2.1.1 (https://github.com/yeikos/js.merge/issues/41 (https://github.com/yeikos/js.merge/issues/41)) that was opened in March, 2021 and has not been closed. So anyone else upgrading their javascript packages needs to be aware of this issue.
Title: Re: Upgrading NodeJS causes gateway.js errors
Post by: Felix on October 03, 2022, 01:10:40 PM
Thanks for the update, it's good to know.
However I noticed the comments in this commit (https://github.com/LowPowerLab/RaspberryPi-Gateway/commit/8da4bcfe5a1315fc7d9ad6f3bc700ebece88c011) that I submitted a while ago.

There was a change to gateway.js that involves calling merge with a false first parameter: merge(false, ...)

Did you get latest gateway.js? That should have fixed your problem even with merge 2.1.1