Author Topic: Database/nodes lost on power loss/improper shutdown [+solution]  (Read 27095 times)

detroit_johnny

  • NewMember
  • *
  • Posts: 35
Database/nodes lost on power loss/improper shutdown [+solution]
« on: September 25, 2015, 03:49:28 PM »
I noticed in working with the gateway a few weeks now that if I update metric.js and I reboot, all my previously programmed nodes come up just as I left them.  This is no problem.

However if my Raspi loses power suddenly without a reboot command, I lose all the nodes and next time I boot up I have to initialize all the nodes again (with node type, label, etc).

Is there a way to save my node configuration on the pi so that it can be restored when that happens?  I am planning to order a MightyBoost or ATXRaspi with a battery so I can get a proper shutdown every time in the future.  In the meantime what options are there?  Thanks in advance.

John
« Last Edit: November 24, 2015, 08:02:12 PM by Felix »

Felix

  • Administrator
  • Hero Member
  • *****
  • Posts: 6866
  • Country: us
    • LowPowerLab
Re: Saving node configurations on the gateway
« Reply #1 on: September 27, 2015, 11:03:14 PM »
John,
What you're saying is a bit worrying. In the sense that after any node update, it should be persisted forever, until you change it again on the node page. That's regardless of reboot or shutdown or pulling the plug (assuming your Pi can still boot after that power fail).
So ... at this point I'd at least recommend trying the latest image if you can. But still, all previous versions were not losing node unexplicably (not that I know of).

detroit_johnny

  • NewMember
  • *
  • Posts: 35
Re: Saving node configurations on the gateway
« Reply #2 on: September 30, 2015, 07:24:57 PM »
I tried another experiment today.

Case 1:  sudo reboot now --> works fine, system comes back up and the nodes are just as I left them.

Case 2:  sudo shutdown now --> wait about 30 seconds, then I unplug, and plug power back in.  All the nodes are gone.  I am using the latest image v6, and I'm pretty sure this was also happening in v5.  This is so strange, I'm not sure what I'm doing wrong??

Felix

  • Administrator
  • Hero Member
  • *****
  • Posts: 6866
  • Country: us
    • LowPowerLab
Re: Saving node configurations on the gateway
« Reply #3 on: October 01, 2015, 08:34:05 AM »
Rebooting or shutting down and unplugging power has nothing to do with how the engine stores data. So what you are reporting is completely strange. I usually use sudo halt and sudo reboot and have never seen anything like this.
After a 'missing nodes' situation, what do you see in the /moteino/db subdirectory?
There should be a gateway.db and a bunch of binary files that store the graph data. There is no process in my gateway that deletes or purges those files or their content.

detroit_johnny

  • NewMember
  • *
  • Posts: 35
Re: Saving node configurations on the gateway
« Reply #4 on: October 01, 2015, 09:49:09 PM »
Hi Felix, I used the "sudo halt" this time.  Guess what?  The nodes are all still in place! 

Still, I was curious why "sudo shutdown now" or a loss of power causes the issue (I guess sudo shutdown now is about the same as an instant power loss, I'm still learning Linux).  So... I took a screen shot of the /moteino/db subdirectory before and after I used that command instead.  I attached the results.  As expected, the nodes all disappeared with the "sudo shutdown now" command, which is the same thing I experience if power is suddenly cut from the gateway Pi before a proper shutdown --> now defined as "sudo halt" or "sudo reboot now".

You will see from the comparison of the screen shots that gateway.db file size has gone to zero, despite all the other files still being in place.






Felix

  • Administrator
  • Hero Member
  • *****
  • Posts: 6866
  • Country: us
    • LowPowerLab
Re: Saving node configurations on the gateway
« Reply #5 on: October 01, 2015, 11:09:00 PM »
Again, strange. At any time during the gateway's activity, if you download the gateway.db, you should see the content being updated as nodes send data. Basically neDB opens the file and always appends to it, even if it's an update or delete. Then once per day my script recycles that DB to reclaim all the stale entries in that file. Anyway that file should always have content. Now perhaps there is a handle to that file by neDB when power is cut or sudo shutdown now is used and it doesn't get closed properly and hence the file ends up corrupted or somehow empty. That I have not verified or tried, but that would be my best guess based on your findings.

Will need to look further into if/why sudo shutdown now causes this problem.
In the meanwhile use sudo halt instead, and perhaps a MightyBoost to keep your Pi juicy during a power outage :)

detroit_johnny

  • NewMember
  • *
  • Posts: 35
Re: Saving node configurations on the gateway
« Reply #6 on: October 02, 2015, 10:23:22 PM »
Yes I agree, my lesson learned is to use the sudo halt or sudo reboot now.  I think you helped me answer my own question too about a backup... I believe I need to save a copy of the gateway.db file in a worse-case scenario.  However as a Linux newbie I can't seem to figure out how to stop the whole node server and moteino code on the next startup from the command line so I can restore the gateway.db file.  I tried the command killall node but that only stops the thing for maybe 3 seconds before it starts back up again on its own.  Not enough time for me to delete the new gateway.db file and place the backup file in its place.  Any ideas?  Do I need to write a script or something to do all of that quickly?

BTW I guess the command sudo shutdown now must not properly close all open files in the system which I think is why gateway.db goes blank.  But the other commands do close any open files first before losing system power.  Maybe??

detroit_johnny

  • NewMember
  • *
  • Posts: 35
Re: Saving node configurations on the gateway
« Reply #7 on: October 02, 2015, 11:45:13 PM »
I noticed if I write an invalid event in the metrics.js file I can make the moteino system not start back up again.  At that point I can delete the gateway.db file and replace it with my saved backup gateway.db file.  After that I comment out or delete my crummy line of code inside metrics.js, do a sudo reboot now and my old nodes and descriptions are back in place   :)

detroit_johnny

  • NewMember
  • *
  • Posts: 35
Re: Saving node configurations on the gateway
« Reply #8 on: October 03, 2015, 10:37:14 PM »
As I mentioned I'm a newb so it took me another 24 hours to figure out that from the terminal I can also issue a sudo stop gateway and then I can put my backup gateway.db file back in place.  After that simply restart the home automation server with sudo start gateway.  Learning new things everyday...

By the way Felix I got a MightyBoost up and running today for my home automation system, using it with my Pi2.  Wondering there must be a way to have a Moteino with a radio on it acting as both the controller for the MightyBoost and as the gateway for the home automation Pi?  Has anyone done this yet and have a sketch they could share?

Felix

  • Administrator
  • Hero Member
  • *****
  • Posts: 6866
  • Country: us
    • LowPowerLab
Re: Saving node configurations on the gateway [solved]
« Reply #9 on: October 05, 2015, 09:22:32 AM »
See this project where [Chris] seems to have combined the MightyBoost control sketch with a gateway receiver sketch.
Code here.

detroit_johnny

  • NewMember
  • *
  • Posts: 35
Re: Saving node configurations on the gateway [solved]
« Reply #10 on: October 05, 2015, 10:20:33 PM »
Thank you Felix, the links you posted look very promising, I should be able to use them!  I'll be unable to study it for a week or so but I will get to it for sure and let you and the forums know if I was able to implement my own version of it and share code.  Thank you much! 

Lensdigital

  • Full Member
  • ***
  • Posts: 155
    • Lensdigital
Re: Saving node configurations on the gateway [solved]
« Reply #11 on: October 06, 2015, 02:23:06 PM »
Just FYI, I had same issue today. Lost all my node names, I think Pi crashed or lost power recently...

Felix

  • Administrator
  • Hero Member
  • *****
  • Posts: 6866
  • Country: us
    • LowPowerLab
Re: Saving node configurations on the gateway [solved]
« Reply #12 on: October 06, 2015, 04:00:27 PM »
Ok so it seems this might be an issue for gateways that don't have any power backup. I'd never would have noticed since I run mine on MightyBoost so when power was lost the backup battery kept it going. I will have to look further into it, but it will require a new setup since it's too difficult to disassemble my gateway as it is now.

Felix

  • Administrator
  • Hero Member
  • *****
  • Posts: 6866
  • Country: us
    • LowPowerLab
Re: Saving node configurations on the gateway [solved]
« Reply #13 on: October 08, 2015, 04:53:03 PM »
I've tried replicating but I can't seem to. The nodes are still there after a sudo shutdown now.
My gateway is using an older image, I will try to test using the official image when I get some extra time.

syrinxtech

  • Sr. Member
  • ****
  • Posts: 347
  • Country: us
    • Syrinx Technologies
Re: Saving node configurations on the gateway [solved]
« Reply #14 on: October 09, 2015, 03:18:43 PM »
Folks, I can also attest to the fact shutting down the Pi unexpectedly causes a total loss of nodes.  I have duplicated this several times including today when I just started up the gateway.  I had 3 nodes saved but when I shut down the Pi I unplugged it without shutting it down properly to simulate a power loss.  When I booted up today I have a gateway.db at zero bytes.

I've been a Unix user for almost 30 years and I've always used "sudo shutdown (-r|-h) now" without issue.  I can't say for sure if it's the shutdown command or the simple loss of power that corrupts the file but it definitely happens.  If it is as simple as shutting down a new way that will take care of 50% of the potential problems.