New Event - MQTT Publish

[kni]

Since there has been interest in MQTT in the forums from time to time, I thought I'd post the following with the hope others will find it useful.

I just finished writing the following event moments ago (i.e. beware hardly tested code  ).
Here is how to use it:

• install the mqtt npm module
• Paste the code below into a js file in the metrics subfolder
• Edit the mqtt.connect line to point to the mqtt broker you want to use
• Restart your gateway
• Add the new MQTT Publish event to the nodes you want to publish to the broker

Once the event is enabled, node statistics will be published each time your moteino checks into the gateway.

Code Select Expand

// Requires the mqtt npm package
// npm install mqtt --save 

exports.events = {
  MQTTPublish : { label:'MQTT Publish', icon:'comment', descr:'Publish the current node\'s metrics on update',
    serverExecute:function(node) {
      const mqtt = require('mqtt')
      // Set the url for the mqtt broker
      // For options such as username and password, see https://github.com/mqttjs/MQTT.js#client
      const client = mqtt.connect('mqtt://hostname')

      client.on('connect', () => {
        client.publish(node.label +'/Updated', node.updated.toString());
        client.publish(node.label +'/Type', node.type.toString());
        client.publish(node.label +'/Description', node.descr.toString());
        // Loop through all the current Node's metrics and publish them
        Object.keys(node.metrics).forEach(function(metric) {
          client.publish(node.label +'/'+ metric, node.metrics[metric].value.toString());
        });
        client.end();
      })
    }
  },
}

[Lukapple]

Thanks for the example.
Did you have any problems with this solution?
I want to forward some events to the Home Assistant and this is probably the right way to do it.

[kni]

This has been working well for me, for the few nodes I need to publish.

There is a design consideration, however. The code currently creates, then closes, an MQTT connection with each attempt to publish each node. This is inefficient, especially for many nodes.

Also, I've read elsewhere that some applications like Home Assistant can detect with the publisher disconnects and will generate an error or warning indicating as such.

I don't use HA, but let me know if you see something like that.  I can rewrite this to make just one connection upon startup of the gateway. The MQTT library would then maintain a single connection to the broker. I just have not yet had a good enough excuse to do the work.

[Lukapple]

I can rewrite this to make just one connection upon startup of the gateway. The MQTT library would then maintain a single connection to the broker. I just have not yet had a good enough excuse to do the work.

This would be great.

However I didn't test it yet with the HomeAssistant. I just broke something in my gateway... it's probably a good time to do a clean install

[Lukapple]

Update: This works fine in home assistant. However we need to refactor this code, to keep the connection online.

[ssmall]

What flavor of MQTT are you guys using?

[kni]

I am using Mosquitto mqtt broker running unauthenticated on a local server, but the node mqtt library is capable of connecting to any kind of broker whether local, in the cloud, with authentication, ssl, etc.

I don't have access to a running instance of the IoT Gateway to test my code, but the following is what I have in mind for something that only makes 1 connection to the broker and stays connected:

Code Select Expand

// Requires the mqtt npm package
// npm install mqtt --save 

var reportMQTTErr = true;
const mqtt = require('mqtt');
// Set the url for the mqtt broker
// For options such as username and password, see https://github.com/mqttjs/MQTT.js#client
const client = mqtt.connect('mqtt://localhost');

client.on("connect",function(){
  console.log("MQTT client successfully connected to broker");
  reportMQTTErr = true;
})

// To avoid repeated errors, set reportMQTTErr to false after the first error
client.on("error",function(error){
  if (reportMQTTErr) console.log("MQTT client can't connect due to: " + error);
  reportMQTTErr = false;
});

exports.events = {
  MQTTPublish : { label:'MQTT Publish', icon:'comment', descr:'Publish the current node\'s metrics on update',
    serverExecute:function(node) {
      if (client.connected){
        if (node.updated) client.publish(node.label +'/Updated', node.updated.toString());
        if (node.type) client.publish(node.label +'/Type', node.type.toString());
        if (node.descr) client.publish(node.label +'/Description', node.descr.toString());
        // Loop through all the current Node's metrics and publish them
        Object.keys(node.metrics).forEach(function(metric) {
          client.publish(node.label +'/'+ metric, node.metrics[metric].value.toString());
        });
      } else {
		console.log("ERROR: MQTT client not connected");
      }
    }
  },
}

The node mqtt library is supposed to handle re-connects automatically.

This is completely untested at the moment. I don't expect it to work right until I debug it.

[Lukapple]

I am using Mosquitto mqtt broker running unauthenticated on a local server, but the node mqtt library is capable of connecting to any kind of broker whether local, in the cloud, with authentication, ssl, etc.

Same here.

I'm not very familiar with NodeJS, but if node instance keeps all those js scripts in memory, then this solution with constant should work. Will try it later.

What will process.exit(1)}) do? Will it stop the execution of whole gateway, or just this event?

[kni]

What will process.exit(1)}) do? Will it stop the execution of whole gateway, or just this event?

The intent behind process.exit is, upon error, stop the node mqtt library from continually trying to reconnect and filling up your logs with the same error message. I *think* this will stop just the mqtt process and not the whole gateway.

[Felix]

Depending on what process runs what. But If you run that in the general context of the gateway.js application, it will stop the whole gateway.
If the restart service is running, it will restart the gateway soon after.

You can also find this in gateway.js, which is used to "restart" the gateway app, by means of stopping it, and letting the restart daemon kick it off again:

Code Select Expand

  socket.on('PROCESSEXIT', function () {
    console.info('PROCESS EXIT REQUESTED from ' + address);
    process.exit();
  });

[kni]

Hmmmm... ok, maybe we should call client.end() instead. That should stop further failed attempts to connect. I've made the change in the above code.

[kni]

UPDATE: So here I was thinking "Gee, I wish I had access to my pi gateway so I could test this new MQTT code". That's when I realized Felix already solved this problem... there is a simulator built right into the gateway, under Log/Terminal.

Right. Now I'm in business.

I made a few changes and have verified it will:
- make just one persistent connection to the broker, even when publishing multiple nodes
- will attempt to autoreconnect to the broker after a connection failure
- will only log 1 error message when a disconnect occurs, instead of spamming the logs every second

I have updated the code in my previous post to reflect this.

Please test.

[Lukapple]

Great, thanks. I've installed the latest version of the MQTTPublish event script.
I've added the retain flag, so I don't have to wait for event to happen.
(This option specifies whether retained messages are sent when the subscription is established.)

Code Select Expand

const mqttOptions = { retain:true }
...
        if (node.updated) client.publish(node.label +'/Updated', node.updated.toString(), mqttOptions);
        if (node.type) client.publish(node.label +'/Type', node.type.toString(), mqttOptions);
        if (node.descr) client.publish(node.label +'/Description', node.descr.toString(), mqttOptions);
        // Loop through all the current Node's metrics and publish them
        Object.keys(node.metrics).forEach(function(metric) {
          client.publish(node.label +'/'+ metric, node.metrics[metric].value.toString(), mqttOptions);

Works fine with home assistant. I'll let you know if I'll find any bugs

@Felix, you should include that mqtt event in the _LowPowerLab metrics

[Felix]

What would the entire set of changes look like?

[Lukapple]

What would the entire set of changes look like?

I've put this script into my metrics subfolder:
mqttEvent.js:

Code Select Expand

var reportMQTTErr = true;
const mqtt = require('mqtt');
// Set the url for the mqtt broker
// For options such as username and password, see https://github.com/mqttjs/MQTT.js#client
const client = mqtt.connect('mqtt://xxx.xxx.xxx.xxx:1883') //This should go to the seperate settings file
const mqttOptions = { retain:true }

client.on("connect",function(){
  console.log("MQTT client successfully connected to broker");
  reportMQTTErr = true;
})

// To avoid repeated errors, set reportMQTTErr to false after the first error
client.on("error",function(error){
  if (reportMQTTErr) console.log("MQTT client can't connect due to: " + error);
  reportMQTTErr = false;
});

exports.events = {
  MQTTPublish : { label:'MQTT Publish', icon:'comment', descr:'Publish the current node\'s metrics on update',
    serverExecute:function(node) {
      if (client.connected) {
        if (node.updated) client.publish(node.label +'/Updated', node.updated.toString(), mqttOptions);
        if (node.type) client.publish(node.label +'/Type', node.type.toString(), mqttOptions);
        if (node.descr) client.publish(node.label +'/Description', node.descr.toString(), mqttOptions);
        // Loop through all the current Node's metrics and publish them
        Object.keys(node.metrics).forEach(function(metric) {
          client.publish(node.label +'/'+ metric, node.metrics[metric].value.toString(), mqttOptions);
        });
      } else {
		    console.log("ERROR: MQTT client not connected");
      }
    }
  },
}

Maybe you could include it into "metrics/_LowPowerLab".

It's really useful to get the data into HomeAssistant - see example in attachment, 90% of those sensors are Moteino (GarageMote, Temperature sensors, MailMote,...).