Since there has been interest in MQTT in the forums from time to time, I thought I'd post the following with the hope others will find it useful.
I just finished writing the following event moments ago (i.e. beware hardly tested code ;D).
Here is how to use it:
- install the mqtt npm module
- Paste the code below into a js file in the metrics subfolder
- Edit the mqtt.connect line to point to the mqtt broker you want to use
- Restart your gateway
- Add the new MQTT Publish event to the nodes you want to publish to the broker
Once the event is enabled, node statistics will be published each time your moteino checks into the gateway.
// Requires the mqtt npm package
// npm install mqtt --save
exports.events = {
MQTTPublish : { label:'MQTT Publish', icon:'comment', descr:'Publish the current node\'s metrics on update',
serverExecute:function(node) {
const mqtt = require('mqtt')
// Set the url for the mqtt broker
// For options such as username and password, see https://github.com/mqttjs/MQTT.js#client
const client = mqtt.connect('mqtt://hostname')
client.on('connect', () => {
client.publish(node.label +'/Updated', node.updated.toString());
client.publish(node.label +'/Type', node.type.toString());
client.publish(node.label +'/Description', node.descr.toString());
// Loop through all the current Node's metrics and publish them
Object.keys(node.metrics).forEach(function(metric) {
client.publish(node.label +'/'+ metric, node.metrics[metric].value.toString());
});
client.end();
})
}
},
}
Thanks for the example.
Did you have any problems with this solution?
I want to forward some events to the Home Assistant and this is probably the right way to do it.
This has been working well for me, for the few nodes I need to publish.
There is a design consideration, however. The code currently creates, then closes, an MQTT connection with each attempt to publish each node. This is inefficient, especially for many nodes.
Also, I've read elsewhere that some applications like Home Assistant can detect with the publisher disconnects and will generate an error or warning indicating as such.
I don't use HA, but let me know if you see something like that. I can rewrite this to make just one connection upon startup of the gateway. The MQTT library would then maintain a single connection to the broker. I just have not yet had a good enough excuse to do the work.
Quote from: kni on May 28, 2022, 08:19:16 AM
I can rewrite this to make just one connection upon startup of the gateway. The MQTT library would then maintain a single connection to the broker. I just have not yet had a good enough excuse to do the work.
This would be great.
However I didn't test it yet with the HomeAssistant. I just broke something in my gateway... it's probably a good time to do a clean install :(
Update: This works fine in home assistant. However we need to refactor this code, to keep the connection online. ;)
What flavor of MQTT are you guys using?
I am using Mosquitto mqtt broker running unauthenticated on a local server, but the node mqtt library is capable of connecting to any kind of broker whether local, in the cloud, with authentication, ssl, etc.
I don't have access to a running instance of the IoT Gateway to test my code, but the following is what I have in mind for something that only makes 1 connection to the broker and stays connected:
// Requires the mqtt npm package
// npm install mqtt --save
var reportMQTTErr = true;
const mqtt = require('mqtt');
// Set the url for the mqtt broker
// For options such as username and password, see https://github.com/mqttjs/MQTT.js#client
const client = mqtt.connect('mqtt://localhost');
client.on("connect",function(){
console.log("MQTT client successfully connected to broker");
reportMQTTErr = true;
})
// To avoid repeated errors, set reportMQTTErr to false after the first error
client.on("error",function(error){
if (reportMQTTErr) console.log("MQTT client can't connect due to: " + error);
reportMQTTErr = false;
});
exports.events = {
MQTTPublish : { label:'MQTT Publish', icon:'comment', descr:'Publish the current node\'s metrics on update',
serverExecute:function(node) {
if (client.connected){
if (node.updated) client.publish(node.label +'/Updated', node.updated.toString());
if (node.type) client.publish(node.label +'/Type', node.type.toString());
if (node.descr) client.publish(node.label +'/Description', node.descr.toString());
// Loop through all the current Node's metrics and publish them
Object.keys(node.metrics).forEach(function(metric) {
client.publish(node.label +'/'+ metric, node.metrics[metric].value.toString());
});
} else {
console.log("ERROR: MQTT client not connected");
}
}
},
}
The node mqtt library is supposed to handle re-connects automatically.
This is completely untested at the moment. I don't expect it to work right until I debug it.
Quote from: kni on May 31, 2022, 07:01:31 PM
I am using Mosquitto mqtt broker running unauthenticated on a local server, but the node mqtt library is capable of connecting to any kind of broker whether local, in the cloud, with authentication, ssl, etc.
Same here.
I'm not very familiar with NodeJS, but if node instance keeps all those js scripts in memory, then this solution with constant should work. Will try it later.
What will process.exit(1)}) do? Will it stop the execution of whole gateway, or just this event?
Quote from: Lukapple on June 01, 2022, 12:44:43 AM
What will process.exit(1)}) do? Will it stop the execution of whole gateway, or just this event?
The intent behind process.exit is, upon error, stop the node mqtt library from continually trying to reconnect and filling up your logs with the same error message. I *think* this will stop just the mqtt process and not the whole gateway.
Depending on what process runs what. But If you run that in the general context of the gateway.js application, it will stop the whole gateway.
If the restart service is running, it will restart the gateway soon after.
You can also find this in gateway.js, which is used to "restart" the gateway app, by means of stopping it, and letting the restart daemon kick it off again:
socket.on('PROCESSEXIT', function () {
console.info('PROCESS EXIT REQUESTED from ' + address);
process.exit();
});
Hmmmm... ok, maybe we should call client.end() instead. That should stop further failed attempts to connect. I've made the change in the above code.
UPDATE: So here I was thinking "Gee, I wish I had access to my pi gateway so I could test this new MQTT code". That's when I realized Felix already solved this problem... there is a simulator built right into the gateway, under Log/Terminal.
Right. Now I'm in business.
I made a few changes and have verified it will:
- make just one persistent connection to the broker, even when publishing multiple nodes
- will attempt to autoreconnect to the broker after a connection failure
- will only log 1 error message when a disconnect occurs, instead of spamming the logs every second
I have updated the code in my previous post to reflect this.
Please test.
Great, thanks. I've installed the latest version of the MQTTPublish event script.
I've added the retain flag, so I don't have to wait for event to happen.
(This option specifies whether retained messages are sent when the subscription is established.)
const mqttOptions = { retain:true }
...
if (node.updated) client.publish(node.label +'/Updated', node.updated.toString(), mqttOptions);
if (node.type) client.publish(node.label +'/Type', node.type.toString(), mqttOptions);
if (node.descr) client.publish(node.label +'/Description', node.descr.toString(), mqttOptions);
// Loop through all the current Node's metrics and publish them
Object.keys(node.metrics).forEach(function(metric) {
client.publish(node.label +'/'+ metric, node.metrics[metric].value.toString(), mqttOptions);
Works fine with home assistant. I'll let you know if I'll find any bugs
@Felix, you should include that mqtt event in the _LowPowerLab metrics ;)
What would the entire set of changes look like?
Quote from: Felix on June 14, 2022, 08:57:23 PM
What would the entire set of changes look like?
I've put this script into my metrics subfolder:
mqttEvent.js:
var reportMQTTErr = true;
const mqtt = require('mqtt');
// Set the url for the mqtt broker
// For options such as username and password, see https://github.com/mqttjs/MQTT.js#client
const client = mqtt.connect('mqtt://xxx.xxx.xxx.xxx:1883') //This should go to the seperate settings file
const mqttOptions = { retain:true }
client.on("connect",function(){
console.log("MQTT client successfully connected to broker");
reportMQTTErr = true;
})
// To avoid repeated errors, set reportMQTTErr to false after the first error
client.on("error",function(error){
if (reportMQTTErr) console.log("MQTT client can't connect due to: " + error);
reportMQTTErr = false;
});
exports.events = {
MQTTPublish : { label:'MQTT Publish', icon:'comment', descr:'Publish the current node\'s metrics on update',
serverExecute:function(node) {
if (client.connected) {
if (node.updated) client.publish(node.label +'/Updated', node.updated.toString(), mqttOptions);
if (node.type) client.publish(node.label +'/Type', node.type.toString(), mqttOptions);
if (node.descr) client.publish(node.label +'/Description', node.descr.toString(), mqttOptions);
// Loop through all the current Node's metrics and publish them
Object.keys(node.metrics).forEach(function(metric) {
client.publish(node.label +'/'+ metric, node.metrics[metric].value.toString(), mqttOptions);
});
} else {
console.log("ERROR: MQTT client not connected");
}
}
},
}
Maybe you could include it into "metrics/_LowPowerLab".
It's really useful to get the data into HomeAssistant - see example in attachment, 90% of those sensors are Moteino (GarageMote, Temperature sensors, MailMote,...).
And require('mqtt') requires ... the mqtt package to be installed during install then?
Yes, this code requires the node mqtt package.
There are a ton of client options the mqtt library supports such as username, password, tls, self signed cert support, reconnect time, qos, the list goes on.
https://github.com/mqttjs/MQTT.js#client
Personally I don't mind configuring the options directly in the .js file, but if Felix is going to officially support mqtt publishing, then it would make sense to add some of these to the settings menu, so the masses can more easily enable this feature. If Felix is willing and we can all agree on what options to support, I can do the work and submit a pr to the github repo. Thoughts?
Or you can add instructions how to manually install the mqtt dependency package of user wants to use the mqtt event. I personally think that mqtt odd a must in IoT world.
I prefer not to fill the settings page with anything that is specific to a package required in a metrics file, at least at the moment.
Also, any require() in metrics file would break the application if that package is not already installed when the app starts up.
This is the only time MQTT has come up, and I can see how it can be useful. If there is a way to keep it as an add-on and not baked into the main app core, then I would be very open to include it.