Sending data from gateway to mysql

Started by Sergegsx, October 25, 2015, 10:18:09 AM

Sergegsx

Hello everyone,

I am starting with all the moteino concept and for now I have been writing a python script to perform the same functionalities as the original gateway.js. Mainly to set and objective and learn doing it.

I have always had my nodes call a php that would do some logic and then store the data in MySQL. I have a full system feeding graphs, alerts and such.

What are your opinions on adding mysql INSERT to the gateway functionality? I am not talking about adding it to gateway.js provided by Felix, but just a general opinion. I really like the save to text file and I have it already working and it is really fast! I set it up so that the Rasberry will save the files in my Synology NAS to prevent SD card degradation.

The problem I see right now is that timing the query for the MYSQL INSERT takes from 0.5sec to 1.2seconds. This cant run in a main thread as we would loose other incoming messages, so I am looking into python multithreading, without much success so far.


spark

I'm considering some changes also.  As much as I like the simplicity of what Felix built, and the fact that it all runs on a stand-alone pi, I'm already having some problems.  I've lost the data several times as others have reported.  Unless a shutdown is done in exactly the right sequence the database is lost.

I'm considering a couple options:

- keep gateway.js largely in-tact.  Rather than store the data in the local NeDB database, I'll store it in a remote mongodb or redis database running on my home server.  Since I already have the home server (I am running open media vault - debian linux), why go through the hassle of installing these databases from source on the pi.  The web application stays, and the gateway.js can be modified to get data from the remote database for reporting and status.

- remove the web application all together, & change gateway.js to be a little more of a stand-alone API, keeping socket.io.  As it is, it is pretty tightly bound to the way the UI works.  I'd remove all local storage.  In this case, I'll run a meteorjs web application from my home server (database in mongodb).