Database/nodes lost on power loss/improper shutdown [+solution]

[Felix]

For now the workaround is to use sudo halt
This issue is definitely on my radar.

[dirkdeb]

Hi I am getting this same problem with power failure and shutdown -h now  or shutdown -h +2
Today I had a few power failures and now if I trie to start /etc/init.d/_nodejs-moteino.sh the system report a datastore.js:67
Error: ENOSPC , write error
I noticed in /moteino/db a file gateway.db ~~  along with the gateway.db file
I tried to restart the gateway again and now this file is missing but now there are  two files gateway_nonmatches.db~ with user:group 'pi:pi' and gateway_nonmatches.db~~ with user:group 'root:root'
Although my one sensor with 5 metrics(s) files are in the db directory and previously was registered in the UI there were no sensors/metrics to be seen before the power loss this morning.
Hope this helps in tracing/catching the culprit.
I start digging to make a plan with this. Maybe one can save/backup the crucial files on change and recover on startup ?!
Thanx

[Felix]

dirkdeb, thanks for your details.
If any of the metrics are logged, can you check (if not too late) whether the metrics log files are still present? They are separate from the general node info DB (gateway.db).

[dirkdeb]

Hi, yes they were still intact in the /home/pi/moteino/db/ directory. ie: 00x_xx.bin pi:pi  attrib. 644
The db directory sits with ownership pi:pi
settings.js root:root
package.json nobody:root
metrics.js pi:pi
gateway.js pi:pi all same read write access 644
I must confess I am running as root ie. start/stop /etc/init.d/_nodejs-moteino.sh
I changed the "start )" to only log to standard output/console to cache the errors.
#

[dirkdeb]

Hi I started to investigate a way to "robustisize" the whole data integrity issue.
I got hold of a "power pack" / power bank for offline cellphone charging, plugged that in overnight and it's running fine. With that I prevent any possibility of power losses.
I then plugged in a USB stick mount the device over my /home/pi/moteino/db directory and got the data logging done on the USB device. With that the possibility of SD card failure is out of the way.
I stopped my node gateway by hitting "/etc/init.d/_nodejs-moteino.sh stop" a few times and...
After the process stopped  I use "/etc/init.d/_nodejs-moteino.sh  start" to start the gateway again ..........and gone was my only sensor!
The data base directory still have the metrics files in it and looks sound, ownerships are good read/write access are good. But gateway.db shows 0 size ?
command df recons I still have a lot of disk space left on the stick.

[Felix]

This is good to know. So gateway.db is getting wiped out. It's probably a neDB issue. I will need to focus on that. Sorry for the lack of updates on this, I know it's pretty important. I have been tied up in other things (personal and LPL related). But still on my priority list. Anyone else is welcome to contribute of course.

[dirkdeb]

Thanx Felix,
Me/ someone with the same problem will find time somewhere/ somehow to get to the root of this. So many things to do so Few time
Regards

[Felix]

There appears to be a fix in NeDB for this issue, see the change log for version 1.4.1.
I will try to test this today.

[Felix]

Looks like this is a known bug and it is being worked on. The creator of neDB reported that the current version is 1.4.0 and that 1.4.2 is not a stable release and does not include the database purging bug fix. I opened a github issue for this and I will watch this to make sure I update as soon as there is a fix.

[Felix]

UPDATE: Luis Chatriot has released a version (neDB 1.5) that fixes the power-cycle database purging issue. I tested on my own Pi and can confirm the database is not lost any longer. However this latest commit in the github master has yet to be published to NPM so until then, whoever wants to upgrade will need to do the following process manually:

- cd /home/pi/moteino/node_modules/nedb - change folder to nedb package location
- npm update - updates nedb and dependencies
- issue a pkill node to recycle the gateway node app
- npm outdated - this will show your current installed version is up to date (note this screenshot shows WANTED as 1.4.0 because 1.5 wasn't published to NPM yet):