Upgrading NodeJS causes gateway.js errors

[TrickyT]

I upgraded Node to v18.9.0 because another application I run required this version. But now gateway gets the following error:

/home/tef/gateway/gateway.js:834
          if (metricsDef.metrics[metric].regexp.test(match[0]))

TypeError: metricsDef.metrics[metric].regexp.test is not a function
    at /home/tef/gateway/gateway.js:834:49
    at newArguments.<computed> (/home/tef/gateway/node_modules/nedb/lib/executor.js:29:17)
    at Cursor.execFn (/home/tef/gateway/node_modules/nedb/lib/datastore.js:518:14)
    at callback (/home/tef/gateway/node_modules/nedb/lib/cursor.js:126:19)
    at /home/tef/gateway/node_modules/nedb/lib/cursor.js:193:12
    at /home/tef/gateway/node_modules/nedb/lib/datastore.js:329:14
    at async.eachSeries (/home/tef/gateway/node_modules/nedb/node_modules/async/lib/async.js:130:20)
    at /home/tef/gateway/node_modules/nedb/lib/datastore.js:323:11
    at fn (/home/tef/gateway/node_modules/nedb/node_modules/async/lib/async.js:582:34)
    at Immediate.<anonymous> (/home/tef/gateway/node_modules/nedb/node_modules/async/lib/async.js:498:34)
Node.js v18.9.0

I'm a javascript noob, but I did google around a bit and it seems like metricsDef.metrics[metric] needs to be a regular expression.
There are several places in gateway.js where syntax like this is used, but in one place where the if() is failing metricsDef.metrics[metric] is set to "{ name: 'DEBUG', regexp: {}, value: ' ' }" and match[0] equals the string "F:99.21". Can someone please suggest changes to gateway.js to fix this problem.

Thanks!

(EDIT: fix typo)

[Felix]

Hi,
Did you just update node? How about node js and reinstall the packages used by the gateway app?
There is a guide here for that:
https://lowpowerlab.com/guide/gateway/node-npm-updates/
Let me know if this helps.

[TrickyT]

I followed the NodeJS & npm update guide. That seemed to go just fine and created a new directory of node_modules. Unfortunately gateway.js encounters exactly the same error as I described previously.

[Felix]

There is nothing wrong with gateway.js.
It is the metric that is not correct, as you can see in the core.js metrics file, the DEBUG metric does have a valid regex. Please check your metrics files against this.

[TrickyT]

OK, will do. Thanks for the tip. I haven't modified any of my metrics files in well over a year, so perhaps the new version of node js uses a more strict interpretation of some language feature. Will report back after I figure it out.

[TrickyT]

The metrics definitions are all fine. The DEBUG metric does have a valid regexp defined in the core.js module, but after this module is loaded and merge() is called, javascript no longer knows it's a regexp. If I call "console.log(metricsDef.metrics[metric].regexp.toString());" for every metric in core.js and _LowPowerLab/*.js and my local metric definitions, what gets logged to the console every time is "[object Object]" while all the other properties like "name" and "value" display what is expected. What should be getting logged is a string containing the regular expression. The runtime error getting logged when gateway tries to find a matching metric...

    TypeError: metricsDef.metrics.regexp.test is not a function

is because javascript doesn't know the object is a regexp and doesn't think the object has a test() method.

[TrickyT]

Here's a test program to illustrate the problem:

Code Select Expand

var merge = require('merge');
var tmp = require('./metrics/core.js');

global.metricsDef = {};
metricsDef.metrics = merge(true, metricsDef.metrics, tmp.metrics);
for(var metric in metricsDef.metrics)
{
  console.log('metric: ' + metricsDef.metrics[metric].name);
  console.log(metricsDef.metrics[metric].regexp.toString());
}

...which produces the following...

metric: DEBUG
[object Object]
metric: V
[object Object]
metric: RSSI
[object Object]
metric: TXLVL
[object Object]
metric: START
[object Object]
metric: TYPE
[object Object]
metric: P_TXP
[object Object]

EDIT: I know the metric definitions in core.js are read in correctly. It's the merge() that's failing.

[TrickyT]

OK, my issue has been resolved.

When I used npm to re-install all the javascript packages, it chose a newer implementation of the merge module. Newer (broken) version 2.1.1, older (working) version 1.2.1. Both come from https://github.com/yeikos/js.merge. I re-installed the older version from a backup and gateway is working again.

This is a known issue in version 2.1.1 (https://github.com/yeikos/js.merge/issues/41) that was opened in March, 2021 and has not been closed. So anyone else upgrading their javascript packages needs to be aware of this issue.

[Felix]

Thanks for the update, it's good to know.
However I noticed the comments in this commit that I submitted a while ago.

There was a change to gateway.js that involves calling merge with a false first parameter: merge(false, ...)

Did you get latest gateway.js? That should have fixed your problem even with merge 2.1.1