You can now check the new features/fixes in the last release(s) here: https://github.com/LowPowerLab/RaspberryPi-Gateway/releases/tag/v8.10
As always:
- make sure to back up your data before you do an upgrade to ensure there is no loss in case something goes wrong
- if updating your Gateway, make sure to read the release notes so you don't miss important update actions you need to take
- report any issues here or submit a PR in github
Felix,
I recently updated to v8.10.0 and everything is working great except for my implementation of garagePoll that I have defined in the userMetrics directory. I am saving the state of the node in the scheduled event. After the update to 8.1.0 I noticed the node state was no longer getting updated properly. I think I narrowed it down to a change that was made to gateway.js. It looks like the schedule function is also updating the state of the node on line 635. This update is overwriting my changes to the node. Is there anyway for me to make changes to a node without having them overwritten?
Thanks,
-Steve
Steve, thanks for tracking this down, can you please share the code snippets you mentioned?
The db.update at the end is what is overwritten. I tested this by commenting out the update starting at line 635 in gateway.js and my garage poll worked as expected.
Here is my code:
garagePoll: { label:'Garage : POLL', icon:'comment', descr:'Poll Garage Status', nextSchedule:function(nodeAtScheduleTime) { return 300000; }, scheduledExecute:function(nodeAtScheduleTime) { db.findOne({ _id : nodeAtScheduleTime._id }, function (err, nodeRightNow) {
var nowDate = new Date(Date.now());
var nodeUpdated = false;
if (nodeRightNow)
{
/*just emit a log the status to client(s) */
console.log('GARAGE POLL STATUS Node Id: ' + nodeRightNow._id + " Status: " + nodeRightNow.metrics['Status'].value);
if (nodeRightNow.metrics['Status'] &&
nodeRightNow.metrics['Status'].value == 'OPEN') {
/* Only automatically close between 9pm and 6am */
if ((nowDate.getHours() >= 21 || nowDate.getHours() <= 6) && nodeRightNow.metrics['Status'].closeCount < 3) {
if (nodeRightNow.metrics['Status'].openDate == null) {
nodeRightNow.metrics['Status'].openDate = nowDate;
nodeUpdated = true;
};
/* Elapsed time in minutes */
var elapsedTimeMinutes = Math.round(((nowDate - nodeRightNow.metrics['Status'].openDate)) / (60*1000));
console.log('GARAGE SHOULD BE CLOSED ' + elapsedTimeMinutes);
console.log('nodeRightNow.metrics[Status].openDate ' + nodeRightNow.metrics['Status'].openDate);
/* Close the door if open more than 10 minutes */
if (elapsedTimeMinutes >= 10) {
sendMessageToNode({nodeId:nodeRightNow._id, action:'CLS'});
nodeRightNow.metrics['Status'].closeCount++;
nodeRightNow.metrics['Status'].openDate = null;
nodeUpdated = true;
};
console.log('Close Count: ' + nodeRightNow.metrics['Status'].closeCount);
};
};
if (nowDate.getHours() < 21 && nowDate.getHours() > 6 && nodeRightNow.metrics['Status'].openDate) {
nodeRightNow.metrics['Status'].openDate = null;
nodeUpdated = true;
};
/* Reset the close count */
if ((nowDate.getHours() < 21 && nowDate.getHours() > 6 && nodeRightNow.metrics['Status'].closeCount > 0) || nodeRightNow.metrics['Status'].closeCount == null) {
nodeRightNow.metrics['Status'].closeCount = 0;
nodeUpdated = true;
};
if (nodeUpdated) {
db.update({ _id: nodeRightNow._id }, { $set : nodeRightNow}, {}, function (err, numReplaced) { console.log(' ['+nodeRightNow._id+'] DB-Updates:' + numReplaced);});
};
};
});
What are these variables?
nodeRightNow.metrics['Status'].closeCount++;
nodeRightNow.metrics['Status'].openDate = null;
Are they the ones being overwritten? Or what part of the node/metrics is being overwritten/removed?
Do you have a log section that shows the sequence of your code writing what you want to the DB, then the new db.update at line 635 (https://github.com/LowPowerLab/RaspberryPi-Gateway/blob/f1bd4357728e3a3287507425723debbc563aea2b/gateway.js#L635) overwriting that?
That new db.update was added to keep track of the events and be able to display info on them in the UI front end.
I modified the garagePoll event code to incorporate more logging to demonstrate what I am experiencing. I also changed gateway.js and inserted a console.log right after the db.update on line 639
var nowDate = new Date(Date.now());
var nodeUpdated = false;
if (nodeRightNow)
{
/*just emit a log the status to client(s) */
console.log('GARAGE POLL STATUS Node Id: ' + nodeRightNow._id + " Status: " + nodeRightNow.metrics['Status'].value);
if (nodeRightNow.metrics['Status'] &&
nodeRightNow.metrics['Status'].value == 'OPEN') {
console.log('Initial nodeRightNow.metrics[Status].openDate ' + nodeRightNow.metrics['Status'].openDate);
/* Only automatically close between 9pm and 6am and closeCount < 5 */
if ((nowDate.getHours() >= 9 || nowDate.getHours() <= 6) && nodeRightNow.metrics['Status'].closeCount < 10) {
if (nodeRightNow.metrics['Status'].openDate == null) {
nodeRightNow.metrics['Status'].openDate = nowDate;
nodeUpdated = true;
};
console.log('After assignment nodeRightNow.metrics[Status].openDate ' + nodeRightNow.metrics['Status'].openDate);
/* Elapsed time in minutes */
var elapsedTimeMinutes = Math.round(((nowDate - nodeRightNow.metrics['Status'].openDate)) / (60*1000));
console.log('GARAGE SHOULD BE CLOSED ' + elapsedTimeMinutes);
console.log('nodeRightNow.metrics[Status].openDate ' + nodeRightNow.metrics['Status'].openDate);
/* Close the door if open more than 10 minutes */
if (elapsedTimeMinutes >= 10) {
sendMessageToNode({nodeId:nodeRightNow._id, action:'CLS'});
nodeRightNow.metrics['Status'].closeCount++;
nodeRightNow.metrics['Status'].openDate = null;
nodeUpdated = true;
};
console.log('Close Count: ' + nodeRightNow.metrics['Status'].closeCount);
};
};
/* Reset the openDate to null when the hours are between 7am and 8pm
if (nowDate.getHours() < 9 && nowDate.getHours() > 6 && nodeRightNow.metrics['Status'].openDate) {
nodeRightNow.metrics['Status'].openDate = null;
nodeUpdated = true;
};
/* Reset the close count */
if ((nowDate.getHours() < 9 && nowDate.getHours() > 6 && nodeRightNow.metrics['Status'].closeCount > 0) || nodeRightNow.metrics['Status'].closeCount == null) {
nodeRightNow.metrics['Status'].closeCount = 0;
nodeUpdated = true;
};
if (nodeUpdated) {
db.update({ _id: nodeRightNow._id }, { $set : nodeRightNow}, {}, function (err, numReplaced) { console.log('gp: ['+nodeRightNow._id+'] DB-Updates:' + numReplaced);});
};
console.log('After save nodeRightNow.metrics[Status].openDate ' + nodeRightNow.metrics['Status'].openDate);
db.findOne({ _id : nodeAtScheduleTime._id }, function (err, findNode) {
console.log('After find findNode.metrics[Status].openDate ' + findNode.metrics['Status'].openDate);
});
};
});
} }
};
Here is the output from the garagePoll:
[2017-10-08T17:14:43.947Z] (sys) Starting
[10-08-17_12:14:51.307] [LOG] **** SCHEDULING EVENT - nodeId:11 event:garagePoll to run in ~1m
[10-08-17_12:14:51.320] [LOG] gateway dbNode.metrics[Status].openDate = null
[10-08-17_12:14:51.347] [LOG] gw: [11] DB-Updates:1
[10-08-17_12:14:54.409] [INFO] AUTHORIZING CONNECTION FROM 127.0.0.1:33644
[10-08-17_12:14:54.424] [INFO] NEW CONNECTION FROM 10.12.20.1
[10-08-17_12:15:51.311] [LOG] **** RUNNING SCHEDULED EVENT - nodeId:11 event:garagePoll...
[10-08-17_12:15:51.315] [LOG] **** SCHEDULING EVENT - nodeId:11 event:garagePoll to run in ~1m
[10-08-17_12:15:51.321] [LOG] GARAGE POLL STATUS Node Id: 11 Status: OPEN
[10-08-17_12:15:51.323] [LOG] Initial nodeRightNow.metrics[Status].openDate null
[10-08-17_12:15:51.324] [LOG] After assignment nodeRightNow.metrics[Status].openDate Sun Oct 08 2017 12:15:51 GMT-0500 (CDT)
[10-08-17_12:15:51.325] [LOG] GARAGE SHOULD BE CLOSED 0
[10-08-17_12:15:51.327] [LOG] nodeRightNow.metrics[Status].openDate Sun Oct 08 2017 12:15:51 GMT-0500 (CDT)
[10-08-17_12:15:51.328] [LOG] Close Count: 4
[10-08-17_12:15:51.330] [LOG] After save nodeRightNow.metrics[Status].openDate Sun Oct 08 2017 12:15:51 GMT-0500 (CDT)
[10-08-17_12:15:51.333] [LOG] gateway dbNode.metrics[Status].openDate = null
[10-08-17_12:15:51.344] [LOG] gp: [11] DB-Updates:1
[10-08-17_12:15:51.348] [LOG] After find findNode.metrics[Status].openDate Sun Oct 08 2017 12:15:51 GMT-0500 (CDT)
[10-08-17_12:15:51.354] [LOG] gw: [11] DB-Updates:1
[10-08-17_12:16:51.317] [LOG] **** RUNNING SCHEDULED EVENT - nodeId:11 event:garagePoll...
[10-08-17_12:16:51.319] [LOG] **** SCHEDULING EVENT - nodeId:11 event:garagePoll to run in ~1m
[10-08-17_12:16:51.332] [LOG] GARAGE POLL STATUS Node Id: 11 Status: OPEN
[10-08-17_12:16:51.334] [LOG] Initial nodeRightNow.metrics[Status].openDate null
[10-08-17_12:16:51.335] [LOG] After assignment nodeRightNow.metrics[Status].openDate Sun Oct 08 2017 12:16:51 GMT-0500 (CDT)
[10-08-17_12:16:51.338] [LOG] GARAGE SHOULD BE CLOSED 0
[10-08-17_12:16:51.340] [LOG] nodeRightNow.metrics[Status].openDate Sun Oct 08 2017 12:16:51 GMT-0500 (CDT)
[10-08-17_12:16:51.342] [LOG] Close Count: 4
[10-08-17_12:16:51.343] [LOG] After save nodeRightNow.metrics[Status].openDate Sun Oct 08 2017 12:16:51 GMT-0500 (CDT)
[10-08-17_12:16:51.346] [LOG] gateway dbNode.metrics[Status].openDate = null
[10-08-17_12:16:51.357] [LOG] gp: [11] DB-Updates:1
[10-08-17_12:16:51.360] [LOG] After find findNode.metrics[Status].openDate Sun Oct 08 2017 12:16:51 GMT-0500 (CDT)
[10-08-17_12:16:51.367] [LOG] gw: [11] DB-Updates:1
At the start of the garagePoll event the OpenDate is always null even though the openDate is successfully updated at the end of the code. closeCount is also not updated as well, but I am not showing it for clarity. gp: is the console.log from when garagePoll updates the node and gw: is the console.log from when the gateway updates the node.
Felix,
Since changing to 8.10 I've been having an issue with some events turning off by themselves. I'm still trying to figure it out but wondered if you had experienced this.
Before going to bed I look to see that all 3 events are on (green) and when I wake up 2 of them are off. I check the log and don't see any "REMOVING SCHEDULED EVENT - nodeId:4" entries that one would normally see.
My userMetrics for the doorbell has not changed since changing from 8.9 to 8.10;
//Door Bell Mote
doorbellSound : { label:'Doorbell : Sound', icon:'audio', descr:'Play sound when doorbell rings', serverExecute:function(node) { if (node.metrics['RING'] && node.metrics['RING'].value == 'RING' && (Date.now() - new Date(node.metrics['RING'].updated).getTime() < 2000)) { io.sockets.emit('PLAYSOUND', 'sounds/doorbell.wav'); }; } },
//doorbellSMS : { label:'Doorbell : SMS', icon:'comment', descr:'Send SMS when Doorbell button is pressed', serverExecute:function(node) { if (node.metrics['RING'] && node.metrics['RING'].value == 'RING' && (Date.now() - new Date(node.metrics['RING'].updated).getTime() < 2000)) { sendSMS('DOORBELL', 'DOORBELL WAS RINGED: [' + node._id + '] ' + node.label + ' @ ' + new Date().toLocaleTimeString()); }; } },
doorbellEmail : { label:'Doorbell : Email', icon:'mail', descr:'Send Email when Doorbell button is pressed', serverExecute:function(node) { if (node.metrics['RING'] && node.metrics['RING'].value == 'RING' && (Date.now() - new Date(node.metrics['RING'].updated).getTime() < 2000)) { sendEmail('DOORBELL', 'DOORBELL WAS RINGED: [' + node._id + '] ' + node.label + ' @ ' + new Date().toLocaleTimeString()); }; } },
doorbellON_AM : { label:'Doorbell ON at 6:30AM!', icon:'clock', descr:'Enable doorbell every morning', nextSchedule:function(node) { return exports.timeoutOffset(6,30); }, scheduledExecute:function(node) { sendMessageToNode({nodeId:node._id, action:'BELL:1'}); } },
doorbellOFF_PM : { label:'Doorbell OFF at 8:00PM!', icon:'clock', descr:'Disable doorbell every evening', nextSchedule:function(node) { return exports.timeoutOffset(20,00); }, scheduledExecute:function(node) { sendMessageToNode({nodeId:node._id, action:'BELL:0'}); } },
doorbellSMSLimiter : { label:'Doorbell : SMS Limited 4/seconds', icon:'comment', descr:'Send SMS when RING is detected, once per 4 seconds',
serverExecute:function(node) {
if (node.metrics['RING'] && node.metrics['RING'].value == 'RING' && (Date.now() - node.metrics['RING'].updated < 2000)) /*check if RING metric exists and value is RING, received less than 2s ago*/
{
var approveSMS = false;
if (node.metrics['RING'].lastSMS) /*check if lastSMS value is not NULL ... */
{
if (Date.now() - node.metrics['RING'].lastSMS > 4000) /*check if lastSMS timestamp is more than 4 seconds ago*/
{
approveSMS = true;
}
}
else
{
approveSMS = true;
}
if (approveSMS)
{
node.metrics['RING'].lastSMS = Date.now();
sendSMS('DOORBELL', 'DOORBELL WAS RINGED: [' + node._id + ':' + node.label + '] @ ' + new Date().toLocaleTimeString()); // line below was un-commented due to getting double DB-Updates entries in log
//db.update({ _id: node._id }, { $set : node}, {}, function (err, numReplaced) { console.log(' ['+node._id+'] DB-Updates:' + numReplaced);}); /*save lastSMS timestamp to DB*/
}
else console.log(' ['+node._id+'] RING SMS skipped.');
Thanks in advance!!
Thanks for the extra details, I will try to look into these issues as soon as I have some time.
additional info;
I created the bell on/off event in the metrics.js file but used your "send SMS" event to see what happens. See the sequence of photos below. Instead of the event just being turned off they are removed from the node.
I must say that is very unusual.
Are you sure your custom code is sound and the update is complete?
Any errors in the gateyway.sys.log or in the browser DevTools?
Well the custom code has been working flawlessly since 8.8 or before.
Not sure if the install is complete but all I can say is I haven't had any other issues except this one.
The update was installed using these steps;
•Copy the contents of this directory in /home/pi/gateway
•run npm install in the /home/pi/gateway directory to install all node dependencies
•Adjust any email/password/SMS settings in settings.json5
No errors in the gateway.sys.log in fact the log doesn't even show the event were removed as mentioned before.
Not sure what DevTools to use to check for errors.
Thanks
sparky,
F12 brings DevTools is the browser (DevTools in chrome, Firebug in Firefox).
It's where you can see the actual DOM of your page, see the traffic and requests the page does, all the scripts and any errors, you can add breakpoints in javascript to step through the code etc.
Anyway I will need to find some time to look into these problems.
Since it's not happening on my end, and nobody else is developing and fixing bugs, it may be a while. I will need to be able to reproduce the problem in my setup to come up with a fix.
Thanks for the details so far.
Well I guess I'm not surprised it's not happening at your end but I posted just in case it was something that was missed.
I don't expect you to waste any more time fixing something that I did wrong. I know you have a lot on your plate and I appreciate the time you spent thus far.
Take care!
Although it doesn't make sense to me its appears that I fixed it. Been running great for a couple days now!
I'm not sure what the problem is yet, but I am also experiencing event issues since 8.10.0. When I add an event, it disappears. If I remove the default "garage poll" event to the garagemote, it comes back on its own. Somehow I have been able to get an event added to my sonarmote, but the garagemote is giving me fits. Has anyone found the cause for this yet? The earlier posts don't seem definitive.
I am aware some folks reported event weirdness since 8.10.
I have not seen those particular issues on my end and I must say the upgrade must be done correctly and completely of all new files to ensure all parts communicate as expected. I would double check there first.
I am also not sure when I will have time to work on a new revision. I have more improvements and features I want to add but I have other important projects going on for a while.
ok I will take a look and see if I can figure it out. I will say that this happens on a brand new build, this was not an upgrade for me. I built a new gateway with a new Raspbian Stretch lite build.
Felix,
Other than following your directions (below) for upgrading, what else can one do to check if it was installed correctly?
If you update from a previous version, make sure to backup your Pi before you update.
This update will require 2 main manual actions from your part (if updating):
•run npm install (should install new suncalc package)
•restart app ( sudo systemctl restart gateway )
sparky,
That's what I usually do and it seems to work just fine. However I always double check that the files are what they should be (time and size). Since the database files didn't change in any way, all I worry about is the index.html for the frontend (and any graphics but those are not critical) and all the backend .js files (including settings.json5).
So if the "Changed files" all have a current date the install should be ok?
Changed files:
•gateway.js
•metrics.js
•settings.json5 (requires manual migration of your own settings)
•package.json ( run npm install for new suncalc package )
•www\images\icon_template.cdr - added battery icon
•lowbattery.png warning icon
•www\index.html (main UI front end)
I did some more testing and un-commented the console logging for the UPDATENODEEVENT event in gateway.js, and I can see when it creates the new event. It just disappears on its own, but I noticed that it happens when the garage poll runs (every 30 seconds). I can even see it create the entry in the DB file when it creates the new event, and also after it gets removed, but I can't see what is actually removing it. It's kind of like it is writing it to the DB but not updating in memory? I don't know. Another interesting thing is I just added a new event to my garagemote, then quickly restarted the gateway service. After the restart, the new event is still there. I noticed in gateway.js that it reads from the dB to determine the events that it needs to schedule, so I thought since it was writing it to the dB that maybe it would persist after a restart. Sure enough it does, but I don't know why yet. Just sharing my findings.
Quote from: rd909 on November 07, 2017, 12:11:14 PM
I did some more testing and un-commented the console logging for the UPDATENODEEVENT event in gateway.js, and I can see when it creates the new event. It just disappears on its own, but I noticed that it happens when the garage poll runs (every 30 seconds). I can even see it create the entry in the DB file when it creates the new event, and also after it gets removed, but I can't see what is actually removing it. It's kind of like it is writing it to the DB but not updating in memory? I don't know. Another interesting thing is I just added a new event to my garagemote, then quickly restarted the gateway service. After the restart, the new event is still there. I noticed in gateway.js that it reads from the dB to determine the events that it needs to schedule, so I thought since it was writing it to the dB that maybe it would persist after a restart. Sure enough it does, but I don't know why yet. Just sharing my findings.
I was not having that issue until I decided to add the "Garage Poll". Deleting the "Garage Poll" and then quickly (under 30 sec's) restart the UI would the issue stop. :(
Ok it sounds like the events needs a serious look.
sparky, can you rephrase this statement, I am not sure I understand the exact order of what's happening (and I assume this is the stock Garage Poll):
Quote from: sparky on November 07, 2017, 07:59:10 PM
Deleting the "Garage Poll" and then quickly (under 30 sec's) restart the UI would the issue stop. :(
Felix,
Sorry for the confusion.
I made a video of the different scenarios to better explain what's going on. At the start of the video you will see two events that I created a few versions back. There has never been any issues like I'm having now when I deleted or added any events. The issue started when I added your stock Garage poll event.
It's a rather long video but I had to wait the 30 sec's between each scenario or action.
https://www.dropbox.com/s/aqhq0mpvy03bq3q/20171108_114414.mp4?dl=0
Thanks
The video makes it easy to see what's going on, thank you for recording it!
So it's the garage poll that's doing something strange. I can reproduce on my end and will look into it.
Thanks Felix,
I don't think it's just the garage poll event. Earlier in this thread I made a video of my doorbell mote with similar issues. Thanks for looking into it.
Hello, all!
Many thanks to Felix for the beautiful Gateway Project.
i installed the gateway by using commands, it's works well. But if i use Pre-built Pi image, the PI-3 can not work, only show a image on the display.
Why? :-*
Quote from: jackkomo on April 26, 2018, 11:10:45 PM
Hello, all!
Many thanks to Felix for the beautiful Gateway Project.
i installed the gateway by using commands, it's works well. But if i use Pre-built Pi image, the PI-3 can not work, only show a image on the display.
Why? :-*
Pre-built Pi image is deprecated as far as I know and script is the way to go
Any older images might not work and are indeed deprecated, are left there for reference. THey might work on older Pis.
Use the setup.sh script to install the latest version of the Gateway.
sparky - the polling event bug should now be fixed, sorry it took so long!
Just get the latest gateway.js, see the commit changeset (https://github.com/LowPowerLab/RaspberryPi-Gateway/commit/5f1db481733f50de76272cdf3137ae1e9cdb42ac) for the actual changes.
I have been experiencing similar problems with the garage polling event deleting other events since I installed the garage mote recently. Eventually I would like to implement a garage polling event that closes the door during certain hours at night if it is open like the one ssmall is using below.
Quote from: ssmall on October 08, 2017, 01:22:57 PM
I modified the garagePoll event code to incorporate more logging to demonstrate what I am experiencing. I also changed gateway.js and inserted a console.log right after the db.update on line 639
I tried updating gateway.js (version 8.10) with the new version released today. I just copy the new gateway.js to the /gateway folder and restart the gateway (sudo systemctl restart gateway) or just reboot. Consistently, the web app does not connect until I put the original gateway.js back. What am I doing wrong? Should I also be updating the other new files on Github? I tried adding the new logUtil.js, metrics.js and package.json, but no luck. I admit I'm in a little over my head.
Unrelated, I have noticed in the past that the metrics.js or files in the userMetrics directory are incredibly touchy and edits frequently cause the web app to not connect. In some cases, I have founds minor errors in the files. In other cases, I make the most minor "correct" edits and the web app won't connect until I put back the original metrics.js.
Can someone please explain how I can view the Gateway CONSOLE on the Pi. I'm assuming this is different from the log file.
To update to the latest non released version (ie last version + all sequential changes which haven't made it into an "official" new release), you need all the .js files, and also the content of the www folder. Usually non released changes are always tested, so if you manually download an replace all the files, you should be good to go, but aware that you are ahead of the last release. If unsure, you can download the last release and use that.
Javascript, like any other computer language, is touchy when it comes to syntax errors and bugs, no way around that.
The client side console LOG in the UI is different than the server side log which is much more comprehensive. However you can log to each individually so they are basically independent of each other.
Quote from: Felix on October 26, 2018, 02:17:39 PM
sparky - the polling event bug should now be fixed, sorry it took so long!
Just get the latest gateway.js, see the commit changeset (https://github.com/LowPowerLab/RaspberryPi-Gateway/commit/5f1db481733f50de76272cdf3137ae1e9cdb42ac) for the actual changes.
Working great so far!
I appreciate your support Felix, thanks
Problems upgrading to 8.11.11. I tried replacing the updated 8.11.1 github versions of gateway.js, logUtil.js, metrics.js, package.json, and the entire userMetrics and www folders from https://github.com/LowPowerLab/RaspberryPi-Gateway. I get the following error in the Console Log:
/home/pi/gateway/gateway.js:49
var port = new serialport(settings.serial.port.value, {baudRate : settings.serial.baud.value});
^
TypeError: serialport is not a function
at Object.<anonymous> (/home/pi/gateway/gateway.js:49:12)
at Module._compile (module.js:409:26)
at Object.Module._extensions..js (module.js:416:10)
at Module.load (module.js:343:32)
at Function.Module._load (module.js:300:12)
at Function.Module.runMain (module.js:441:10)
at startup (node.js:140:18)
at node.js:1043:3
Garage Poll Questions:2. What does the unmodified "Garage Poll" event in metrics.js do. Is it necessary for the basic Garage Mote "status"? What causes it to auto start in the first place, something in gateway.js?
3. How does ssmall's Garage Poll work? I can see generally that it only has the "close" action if the time is between 9pm and 6am and if the "status" has been "CLOSED" for more than 10 minutes. What is "closecount" used for and what does the "nodeRightNow" command do?
4. Is it possible, but maybe not the most functional idea, to create a simpler event that just looks at the current Garage "status" and if it is OPEN and the time is between 9pm and 6am, just CLOSE the door?
Node.jsI guess I finally need to learn node.js and Javascript if I want to understand this better, right? Thanks.
Quote from: Tony Weil on October 27, 2018, 01:19:45 PM
Problems upgrading to 8.11.1
1. I tried replacing the updated 8.11.1 github versions of gateway.js, logUtil.js, metrics.js, package.json, and the entire userMetrics and www folders from https://github.com/LowPowerLab/RaspberryPi-Gateway. I get the following error in the Console Log:
/home/pi/gateway/gateway.js:49
var port = new serialport(settings.serial.port.value, {baudRate : settings.serial.baud.value});
^
TypeError: serialport is not a function
at Object.<anonymous> (/home/pi/gateway/gateway.js:49:12)
at Module._compile (module.js:409:26)
at Object.Module._extensions..js (module.js:416:10)
at Module.load (module.js:343:32)
at Function.Module._load (module.js:300:12)
at Function.Module.runMain (module.js:441:10)
at startup (node.js:140:18)
at node.js:1043:3
Trying to do my homework. It appears that the serialport module has been changed to //https://github.com/node-serialport/node-serialport, version "serialport": "~6.2.1", in the 8.11.1 gateway.js. It used to be //https://github.com/voodootikigod/node-serialport, version "serialport": "~2.0", in 8.10.
Is the change as simple as running "npm update serialport"??
It seems that socket.io, nodemailer and request have also been updated between 8.10 and 8.11.1. Do I need to upgrade those also or will "npm update" upgrade all of the modules? Will this get the versions in the package.json or possibly newer versions in the npm repository, whatever they happen to be?
package.json 8.11.1
serialport": "~6.2.1",
"socket.io": "~2.1.1",
"nedb": "~1.8",
"json5": "~0.5.0",
"nconf": "~0.8.2",
"nodemailer": "~4.6.7",
"console-stamp": "~0.2.4",
"colors": "~1.1.2",
"request": "~2.87",
"merge": "~1.2.0",
"suncalc": "~1.8"
package.json 8.10
"serialport": "~2.0",
"socket.io": "~1.5",
"nedb": "~1.8",
"json5": "~0.5.0",
"nconf": "~0.8.2",
"nodemailer": "~1.10",
"console-stamp": "~0.2.4",
"colors": "~1.1.2",
"request": "~2.78",
"merge": "~1.2.0",
"suncalc": "~1.8"
Quote3. How does ssmall's Garage Poll work? I can see generally that it only has the "close" action if the time is between 9pm and 6am and if the "status" has been "CLOSED" for more than 10 minutes. What is "closecount" used for and what does the "nodeRightNow" command do?
Tony, my version of garage poll tries to handle a few situations. I did not want to close the door as soon as it was detected to be open. My wife gets up early in the morning to work out and starts her car to warm it up in the winter, so I wanted to have a period of time that the door stayed open so carbon monoxide would not build up in the garage. The close count is there in case the vehicle is not pulled all the way into the garage and the door is not continuously closed on the vehicle during the night, the code attempts to close a max of 3 three times. My code used to work before version 8 of the software. It seems (I could be wrong on this) that any changes that are made to a node and saved in an event that is firing are overwritten by the runAndReschedule method. If you are OK with the garage door closing as soon as it is detected to be open then you should be good.
-Steve
Steve,
So is the code you posted still working or did you have to re-write due to the new updates? I would like to try this "auto close" feature myself. All you changed was the new Poll event?
Thanks
Sparky,
The code does not work any more in Version 8.x (it used to work in version 7.x) because I am trying maintain state in the node. The changes that are made to the node inside of the event are saved but they are then overwritten in the runAndReschedule method with a node that does not contain the information that was saved in the event. If you don't try to save any state in the node it should work. You just can't take some of the precautions I was trying to take in the code I posted.
Hope that makes sense,
-Steve
ok, thanks Steve. bummer :-\
Tony,
If you have not recently installed the gateway then you also have to update nodejs packages. I made some updates/changes to the packages in the recent months.
Get the latest packages.json, then run npm update:
cd ~/gateway
sudo npm update
Guide here:
https://docs.npmjs.com/getting-started/updating-local-packages
If all else fails, save your db folder and settings.json5, and run a clean setup again, indeally on the latest release of raspbian.
Thanks Felix. Still learning...
Some success! Eventually, after some problems I'll describe further below, I got the latest package.json and only updated serialport to 6.2.1 using "npm update serialport". Before this, as you suggested, I also copied the following new files/folders from the 8.11.1 update on Github: gateway.js, logUtil.js, metrics.js and the whole www folder.
After running npm update serialport, npm outdated looks like this and version 8.11.1 seems to be running fine, even though some packages were not updated.
pi@raspberrypi:~/gateway $ npm outdated
Package Current Wanted Latest Location
colors 1.1.2 1.1.2 1.3.2 colors
console-stamp 0.2.6 0.2.7 0.2.7 console-stamp
json5 0.5.1 0.5.1 2.1.0 json5
merge 1.2.0 1.2.1 1.2.1 merge
nconf 0.8.5 0.8.5 0.10.0 nconf
nodemailer 1.10.0 4.6.8 4.6.8 nodemailer
request 2.78.0 2.87.0 2.88.0 request
serialport 6.2.2 6.2.2 7.0.2 serialport
socket.io 1.5.1 2.1.1 2.1.1 socket.io
==================
My previous failed attempt was as follows.I would love to understand why this didn't work and how to upgrade the remaining packages, if they actually need it.
With new package.json in place, I ran "npm outdated", which showed a number of packages needed updating, serialport and nodemailer being the most in need of a major update.
pi@raspberrypi:~/gateway $ npm outdated
Package Current Wanted Latest Location
colors 1.1.2 1.1.2 1.3.2 colors
console-stamp 0.2.6 0.2.7 0.2.7 console-stamp
json5 0.5.1 0.5.1 2.1.0 json5
merge 1.2.0 1.2.1 1.2.1 merge
nconf 0.8.5 0.8.5 0.10.0 nconf
nodemailer 1.10.0 4.6.8 4.6.8 nodemailer
request 2.78.0 2.87.0 2.88.0 request
serialport 2.0.6 6.2.2 7.0.2 serialport
socket.io 1.5.1 2.1.1 2.1.1 socket.io
After running "npm update", the web client could not connectand there were errors in the server log about nodemailer. Unfortunately, those server logs got lost. Attached is npm_update.txt that shows what happen while "npm update" was running.
I don't recall having so many warnings, definitely some odd interaction between your nodejs version and the packages dependencies. From your logs and what you show, I cannot tell if there's a clean way to updating your existing setup. I now realize the socket.io was also updated, which includes a client side script in the www folder, so that whole folder should also be updated to latest.
Then nodejs itself might need updating. I made some changes to the setup script, which is a better install of nodejs IMO. I suspect you might spend a lot more time trying to dig yourself out of warnings and error logs. Rather, I would recommend backing up your existing db folder and settings, and getting latest raspbian and doing a clean install, should take no more than 10 minutes if you have a recent Pi. I am in no way an expert on node/npm specific package errors, dependencies and how to quickly resolve them. I usually just wipe and clean install myself, knowing that's generally a good idea and will fetch the latest Pi security updates as well. Let me know what you decide and how it goes...
I really liked the idea of a clean install instead of fixing one problem after another. It was relatively easy and worked like a charm. Thanks for the good advice.
To make sure I didn't kill my working Gateway, I set up a new 8.11.1 gateway on a second Pi with a different IP address and commented out the MightyHat shutdown (shutdownirg.py). Then I copied over the db, settings.json5 and the UserMetrics folder. Once it was working, I just moved the SD card to the Gateway Pi hardware with the MightyHat, changed the IP address and uncommented the MightyHat shutdown.
I temporarily lost a custom mote definition (my LeakMote) that I forgot was defined in metrics.js and had a custom icon in www/images. I was able to restore and move the mote definition to userMetrics.js so I don't lose it on the next upgrade, but I was not able to figure out how to move the icon to the userMetrics folder. I tried all kinds of paths to the icon.png inside the mote definition. I also temporarily lost some custom Events and a some custom Metrics that I had originally put in metrics.js and had never moved over to userMetrics. All fixed now.
Now that I have a working 8.11.1 with a working Garage Poll, I can get back to what I really wanted to do and work on getting the updated Garage Poll event to close the garage door between midnight and 6am.
I will soon release a new version which will include an example garagePoll with auto-close functionality, based on ssmall's work. Stay tuned.
That is really great news, Felix. So far trying to understand the garagePoll with auto-close has been beyond my current ability. Someday, I would like to understand it.
One thought I had for an addition to the GarageMote was a CO sensor. My wife has a vehicle that uses a fob instead of keys so it is really easy to walk away from the vehicle while its still running. It hasn't happened yet but I have seen stories where that has happened. Something to think about.
Quote from: Tony Weil on October 31, 2018, 10:27:02 PM
That is really great news, Felix. So far trying to understand the garagePoll with auto-close has been beyond my current ability. Someday, I would like to understand it.
I will say though that as much as ssmall has put a few "safety" features into his garagePoll event, I would still advise anyone who plans to use it to understand what the event does by looking at the code and comments. Closing the door automatically should be considered very carefully.
ssmall, yes CO is a concern and it's a great suggestion. Keys or fob or start button ... its all the same, if you walk away.
I never looked into CO sensors so if you have proposals please feel free to share (a new thread?).
I agree that one should have some healthy respect for automating a closing garage door and understand what the code is doing in general terms, i.e. closes between 12am and 6am after being open for 10 minutes, don't close if it has tried to close x number of times in x minutes, etc.
I would really like to understand the code in more detail so that I could modify it or write my own in the future. I have had some success at writing simple events, like SMS if indoor temperature too low, by modifying existing events. The Garage Poll script exceeds my current ability, but I'm willing to put in the effort to learn whatever is necessary to be able to do it in the future. Where should I start? Learn more about Javascript and Nodes.js? Can someone explain the current or enhanced GaragePoll event in much more detail for a newbie? I realize it is asking a lot. Thanks in advance!
Folks, V9 is published & ready to be installed.
Guide: https://lowpowerlab.com/guide/gateway/
Repo: https://github.com/LowPowerLab/RaspberryPi-Gateway/
This thread will now be locked.