LowPowerLab Forum
Hardware support => Moteino => Topic started by: Fwessels on June 29, 2014, 01:59:08 AM
-
Hi I would like to use the Moteino in my home automation project and I want to use MQTT as the communication protocol. The great thing about MQTT you can then integrated the Moteino into any open source home automation platform like Openhab, Openremote, PiDome any system that use MQTT. My question is how do I get the Moteino to communicate to MQTT? Please note I am still very new to arduino and programing.
Thanks
Francois
-
I was not aware of MQTT before your post. I've downloaded their specification (MQTT-SN), and it seems the protocol is very clunky. They waste space in every byte they send, requires the gateway to store an unlimited amount of data for the nodes -- forget using a moteino for the GW, and they expect very precise timing between nodes.
Typically, instead of storing a message for a sleeping node, and react when it wakes up, it's up to the node to tell how long it's going to sleep, and the expectation is that both the GW and node have the same clock, so the GW consider the node lost if it can't send the pending msg to the node after the sleeping time. If the node wake up early (interruption, shorter clock, brown out/watchdog), then it has to first ping the GW, pop all the pending messages, and then send the interruption message.
I think you can probably find a library that could compile on Arduino, but it means that the entry point on your wireless network will have to be large.
Depending on the complexity of your intend, you better learn from scratch, and make you simple installation simple, you'll probably get more internal reward from this.
-
I will back bobleponge.
Sounds terribly complicated. Home automation can't be that complicated, in fact i will say it's forbidden to be that complicated.
I want nodes to be as autonomous and flexible as possible.
-
There is a MQTT library for arduino done by Nick Of leary http://knolleary.net/arduino-client-for-mqtt.
I agree it should be easy and simple to integrate devices into home automation system. I just battle see what is the easy way off doing this. There is a lot of talk about the Internet of things and I would like to use the Moteino as my base for this. What I read on your Web site the Moteino is low power and by using the rfm69 I don't have to run any wires to communicate to any of the sensor. Your Moteino must cheaper than the Zwave or Xbee. I like what you have done with the Switchmote. So the question is how do we put this all together in home automation systems from monitoring water usage, power usage trigger events etc.
-
There is a MQTT library for arduino done by Nick Of leary http://knolleary.net/arduino-client-for-mqtt.
I agree it should be easy and simple to integrate devices into home automation system. I just battle see what is the easy way off doing this. There is a lot of talk about the Internet of things and I would like to use the Moteino as my base for this. What I read on your Web site the Moteino is low power and by using the rfm69 I don't have to run any wires to communicate to any of the sensor. Your Moteino must cheaper than the Zwave or Xbee. I like what you have done with the Switchmote. So the question is how do we put this all together in home automation systems from monitoring water usage, power usage trigger events etc.
I am racing towards the same goal. I have a version of this working, but it's not ready for release. It needs to be prettied up, refactored and more functionality added.
-
If you have a fairly powerful gateway (ie Raspberry Pi), perhaps it could run a MQTT front end program that exposes the remote nodes as MQTT devices but actually uses a lighter weight protocol to do the actual wireless communication with the nodes. I agree that MQTT to the remote nodes doesn't make sense when low power is a priority.
-
I came across this web site where he is using MQTT and the RFM69 to communicate to Openhab.
http://electronichamsters.wordpress.com/2014/06/09/home-automation-with-arduino-and-openhab/
Yes you cannot put MQTT on any of the sensors you will have to put it on a gateway of some sort.
-
I am running modified jeenodes (emontx units from openenergymonitor) with mqttsn over rfm12b radios back to openhab. I use the RSMB MQTT broker on a rpi with Rfm12pi module on the gpio. I had to modify a mqtt-sn library for arduino and developed a mote lib as we'll for managing io changes. The mqttsn lib uses the jeenode lib.
I still need to. Upload the code but the wiki is populated: https://github.com/jasonhector/Mqttsn-Mote-Library
I'm buying some moteinos and porting to this hardware platform, but still plan to use mqtt and openhab.
I also am monitoring my energy usage with ct's and using mega2560 and enj Ethernet to publish mqtt messages on my wired Ethernet network. I hacked emoncms from openenergymonitor to receive mqtt messages. Also reading my paradox alarm for events and publishing those via mqtt as well. Openhab is the ultimate receiver of all mqtt messages from the wired, wifi or rfm radio network.
-
Jason, that's pretty cool, thanks. Will be great to follow your progress if you want to share any of that in this forum.
-
Sure, No Problem sharing. Not sure which parts are worth sharing, but Im happy to contribute where possible.
-
Hey Jason - you sound like you have a very similar setup to me. I am running openHAB and EmonCMS. I am just starting with Moteinos to build a cheap/wireless sensor network. I have an Arduino ethernet clone connected to one Moteino (via I2C) which acts as the gateway from the Moteino RF network to my MQTT broker.
I also have an EmonTX shield sitting on another Arduino ethernet clone, which posts MQTT messages for energy readings. I haven't been able to hack EmonCMS however to accept MQTT directly. I have had to use mqttwarn (https://github.com/jpmens/mqttwarn) to transform the MQTT payloads to HTTP GETs for EmonCMS. Would you care to share how you configured EmonCMS?
I am using the PubSubClient (http://knolleary.net/arduino-client-for-mqtt/) for all my Arduino based MQTT publishing.
-
How are you guys reading the energy consumption? As far as I know there are only two ways.
1. Place a sensor over an analog electricity meter's dial and count the turns of that black line
2. Place a magnetic coil (I forgot the name) over the main electricity wires coming from the street.
For 1, I'm struggling to find a sensor that works. Yet to try the ones used by line following robots.
For 2, I'm struggling to risk electrocution by opening the main 200W panel and placing those coils around them.
How are you guys doing it?
-
The guys at OpenEnergyMonitor (http://openenergymonitor.org/emon/emoncms) simply use split core current transducers they place around the mains wires to get current, and they measure the voltage from a 9V transformer. Not very ideal and there are many assumptions, but it gives you a ballpark of what's going on in your house.
To get a REAL reading you need an integrator chip that is dedicated for energy measurement. Such as the analog devices ADE series of chips. I used ADE7758 for 3 phase, ADE7753 and ADE 7763. They work well but are tricky to setup. Also they still use split coils but also work with rogowsky coils. To get accurate readings you have to get more expensive commercial grade split coils. The non-spit coils will always be more accurate, but they are more difficult to install since you have to disconnect the mains cables coming in, insert the coils, then reconnect. Doable, but if you're not mastering mains you should not do it, you can die :)
-
This is the reason that people transduce the dial movement - you need to clamp either line OR neutral to get a reading from your CT, which means modifying your device or digging in your panel (or otherwise getting creative).
C
-
Hi There, I though I would share my "early days" home automation setup:
1. Moteino Sensors (Battery Powered), broadcasting periodic sensor readings
2. Raspberry PI running standard Raspbian OS containing the following:
2a. USB (Serially Connected) Moteino, that relay messages from RFM69 to Serial Port
2b. Node-RED, Just acts as a gateway receiving messages (Serial) from the Moteino sending to MQTT Queue
3. Small Debian “Virtual” Server containing the following:
3a. MQTT (Mosquitto) installed to act as message Broker
3b. Node-RED, receives messages from MQTT, and broadcasts to Emon CMS, although this functionality can be extended to provide any number of functions. This is the main controller.
3c. Emon CMS for displaying data
In my setup, I am using a small Debian “Virtual” Server. The reason is it provides a level of quality of service (backups etc), but everything on here could be run just as easily on the PI
-
PIDome home automation software http://pidome.wordpress.com/ now have support for MQTT. I being play with this for the last couple of weeks and it is really a create product. It is still in development phase however it is stable. I run on the RaspberryPi and is not that resource intensive as OpenHab on the PI. It is also not that difficult to setup and to run :D as compare to OpenHab
-
Hi Felix have you come across this guys MySensors http://www.mysensors.org/? They have a create eco system build around the Arduino and the NRF24L01 wireless. I being play with the NRF2401 as well. The NRF24L01 is not as great as the RFM69 transceiver. It also don't have the range as your Moteino solution. Also it not so compact as you device. To build a sensor and then to integrate this into your home automation system is really easy.
-
I have not, I also tried the nordic radios but there is no range on those things unless you get the much bulkier amplified versions, by that point price is much higher and size much bigger, I will go with RFM69HW instead :)
-
Felix why don't you work with MySensor team to get your Motion integrated with there software API? The current solution do support different radio type but there biggest focus is Nrf24l01 and not on RFM69.
If you look at there eco system they have MQTTgateway or Serial gateway that run on a arduino and wireless radio and from here you can integrate into 6 diffrant controller's and then you have your 20 sensors that you can connect to your home automation systems. The senor library is growing as the user community make recommendations Hek one of the founders of MySensor will look at this and will add the new sensor. Also one of their principle is to keep thing as simple as possible so that people that is not that experience in programme and electronic will be able to use. If I look at your Moteino solution it is great solution for this as you don't even have to fit the radio it come already fitted. So all you have to do is add the sensor you want and you will be up and running within 20 minutes.
-
I contacted Henrik and I'm in exchanges with him, he seems open to the idea. Thanks for the tip.
-
Hi Felix how is it going with the talk with Hek and the inclusion of Moteino and part of the MySensor ecosystem?
-
Not yet, im waiting for updates from Henrik.
-
To be able to use Moteino with MySensor would be GREAT ! :D
-
The guys at OpenEnergyMonitor (http://openenergymonitor.org/emon/emoncms) simply use split core current transducers they place around the mains wires to get current, and they measure the voltage from a 9V transformer. Not very ideal and there are many assumptions, but it gives you a ballpark of what's going on in your house.
Just a quick note - I am using a CT sensor around my mains connection to the grid and monitoring using a sketch from the guys at OpenEnergyMonitor. It takes voltage and current samples continuously (i.e. every mains cycle - 50 times a second) and I am getting daily totals that are less than 1% different to what I am being billed by my electricity provider.
I am finding it an extremely accurate way to monitor my energy usage.
-
Just a quick note - I am using a CT sensor around my mains connection to the grid and monitoring using a sketch from the guys at OpenEnergyMonitor. It takes voltage and current samples continuously (i.e. every mains cycle - 50 times a second) and I am getting daily totals that are less than 1% different to what I am being billed by my electricity provider.
I am finding it an extremely accurate way to monitor my energy usage.
Cool, what microcontroller/transceiver are you using?
-
Apologies - I never received a notification that there was a reply to my post!
I am using an Uno clone with built in ethernet - http://www.freetronics.com/products/etherten - with the Open Energy Monitor EmonTX Arduino Shield v2 attached. The sketch takes readings every mains cycle and publishes MQTT messages every 5s with the cumulative totals for each of the 4 CT sensors attached.
Hasn't missed a beat in a few months now. The Etherten is powered by POE.
-
Hi folks,
Check out the Instructables website for the Uber-Home-Automation project
The openHAB project there explains the full setup including MQTT and has also all the code examples.
MQTT works absolutely fine in that setup.
Look at:
http://www.instructables.com/id/Uber-Home-Automation/
Anton
-
Ok, the first bits is in place now.
https://github.com/mysensors/Arduino/tree/development/libraries/MySensors
Thanks
Henrik
-
Hi Hek this is great news. So is everything working what is still missing?
-
Just merged a pull request and the link above points to our development-branch (unstable).
It hasn't been tested much more than from the contributor so we need help if interested.
Here is a good thread to post feedback:
http://forum.mysensors.org/topic/285/porting-mysensors-to-work-with-the-radiohead-library
-
I wanted to take the time to report SUCCESS! I really like the mysensors protocol and the flexibility it provides. Now that it supports moteinos and RFM69 radios, I think it is the perfect combination.
Using the development branch of mysensors: https://github.com/mysensors/Arduino/tree/development/libraries/MySensors I was able to get a serial gateway and sensor node working by downloading it to my libraries folder. Here are some notes / tips I discovered along the way.
1. Mysenors development branch does not have the latest rfm69 code from Felix, specifically the rx dead lock scenario.
https://github.com/LowPowerLab/RFM69/commit/ed2fd5b8d55d011ed8164d9d517f364cc7841a0c
However it was very easy to update with the latest code.
2. I had to edit the mySensors\MyConfig.h file prior to uploaded my sketch. The file is dead simple, it really only had 2 options. The first is to define a sensor / serial gateway / or MQTT. The second option is for the radio. Obviously RFM69.
3. Next I had to edit the mySensors\MyDriverRF69.h to define all the cool rfm options such as the encrypt key, the network ID, and IS_RFM69HW option.
4. It does not look like the gatewayid or nodeid in this file have any effect. You have to set the nodeID in your main sensor sketch.
5. I then basically implemented the code from this post on the mysensors forum. http://forum.mysensors.org/topic/655/serial-gateway-connection-to-openhab/3. I did not have to make other changes to the serialgateway sketch.
6. In the sensor sketch from the previous post I had to change MySensor gw(48, 49); to MySensor gw;
Out side of that everything was good to go. I wired up a dht, uploaded the sketches and I was off any running.
The next step is to try and get an MQTT gateway up and running. I have a enc28j60 module, and it looks like the only change I would need to make is to the libraries\UIPEthernet\utility\Enc28J60Network.h file so that the CS pin can be redefined since it is already used from the RFM69 radio.
//#define ENC28J60_CONTROL_CS SS
#define ENC28J60_CONTROL_CS 7
#define SPI_MOSI MOSI
#define SPI_MISO MISO
#define SPI_SCK SCK
//#define SPI_SS SS
#define SPI_SS 7
I am not convinced that I need an mqtt gateway. A moteino is small and pretty easy to attach to a raspberypi or beaglebone gateway. There might be scenarios where someone might want the flexibility of a lan port provides. I am not sure what they memory requirements will look like, or if there is a way to secure the MQTT messages. I know that the protocol allows for an ssl type encryption, but I dont know if the arduino library supports it.
Thanks again to Felix, Hek, and everyone on this forum and the mysensors forum of all the great information.
-
HI nicbrink any success with the MQTT gateway?
-
Hi All,
Though I would jump in here with some thoughts and opinions because, well, that's what forums are for ;)
I've been doing a LOT of research the past weeks into the current state of affairs in the home automation industry and the more recent injection of open source community projects and initiatives, which seem to be, indeed, moving and shaping the industry. Nice to see that!
I worked in the IT industry for more than 20 years and finished my career as a systems architect. Fancy title for the guy who figures out how to make crap work ;) But, doing that, and being an engineer, taught me to try to look at things first from a 20,000 foot level and then drill down to find, and use, best practices and technologies that were most likely to achieve wide spread adoption and support, thus likely to endure the longest. This is my observations and opinions...just opinions guys ;)
I see the likes of OpenHAB and PiDome trying to be all things to all people and trying to create a platform that will congregate the data and control of everything that's deployed out there already and everything that is to come in the future. In doing that, within their architectures there will always be "chasing the dog" scenarios in which for every new sensor/node/controller rolled out, a new definition has to be created to fit into the framework. Sorta like computer hardware guys that have to also build drivers for every stinking version of Windows! It will never end as long as every manufacturer out there is using a different protocol. As well, in the case of people like us on this forum who are experimenting with Felix's very cool Moteino's...many are concerned now too with building, or implementing, tools and/or gateways of some sort to translate your custom made protocol into something that can go back and forth with some kind of management/presentation platform which may turn out to be just a "flavor of the day".
I guess it goes without saying that the industry needs standards to move forward in a meaningful way if we are ever to really achieve interoperability like that envisioned in the Internet Of Things (IoT). Manufactures HATE standards because without their proprietary pieces, then they have to directly compete, and be better than, the competition. BUT..as we have seen in many industries, eventually standards are developed and adopted by large and small developers alike and the world becomes a better place. Think TCP/IP...
From what I can see, MQTT is going to become the defacto standard as a protocol for automation. Given that major industry leaders are quietly getting involved in, or buying other companies, that utilize MQTT, that will start to drive it's wide spread awareness, utilization, and implementation. In examining MQTT myself, I believe it has all the needed flexibility and expandability to work well for automation. It's easy to imagine it being structured like SNMP but with a whole lot better security and much lower bandwidth requirements!!
It's also my opinion that manufacturers of devices, and that includes people like us here building cool Mote's, that the MQTT message structure should be pushed all the way back to the firmware on the device. When building your payload message, why not use an MQTT convention? In so doing, all your wireless to IP network gateway has to do is take your payload and forward it on to an MQTT broker. From there, some very nice management and control platforms could be built that only have to deal with getting and sending MQTT messages. That, in my opinion, removes a whole lot of the moving parts from the current paradigm that exists out there. Will it work for existing devices with every manufacturers existing protocols and api's...no. But, if every device manufacturer starts having to implement MQTT in their devices in order to be attractive to consumers and fit into some nice management platforms that will evolve, then they will...or die. That's the way this industry works...over and over and over again.
As I'm developing my Mote's, I will be working towards implementing MQTT message structures in some kind of sensible pattern and see how it all works out. Now, speaking of standards, we all need to structure our MQTT messages in the same format...ok, so we will all do it MY way ok?? LOL. I believe that too will come...again..think something like SNMP and it starts to make sense. Well, it does to me anyway...
-
@BrianB, I've seen the 'fury' behind MQTT and, being a guy who likes to keep things simple, it was something that I purposefully avoided since I can build my own data structures and packet formats, thankyou...
However, after reading your post, I was intrigued by your argument justifying the likelihood of MQTT's success. As a result of your post, I've come out of my shell and, at least, scoped out MQTT (on Wikipedia, of course).
Net: Thanks to you, I now have ONE MORE THING to think about and research! :D
Thanks,
Tom
-
@BrianB, I've seen the 'fury' behind MQTT and, being a guy who likes to keep things simple,
Tom, that is exactly why I think MQTT will be adopted. You will find that after your research (sorry about that) that it truly is a very simple concept and structure that can be easily implemented by manufacturers in firmware. I'm certainly not a sketch expert yet but I can easily envision small, tight, fast code that MQTT formats your payload just as easily, if not easier, then sticking colons between variables ;)
Glad to hear you interested in the discussion at least...
-
As I'm developing my Mote's, I will be working towards implementing MQTT message structures in some kind of sensible pattern and see how it all works out. Now, speaking of standards, we all need to structure our MQTT messages in the same format...ok, so we will all do it MY way ok?? LOL. I believe that too will come...again..think something like SNMP and it starts to make sense. Well, it does to me anyway...
Thanks for your post, and I agree with MQTT as a hub, but think pushing it down to the RFM69 layer is not advisable, especially since one of the main use cases is battery operation, where every byte transmitted costs precious milliamps. I have gone for a very small binary structure, with a Mote with ethernet shield acting as a gateway to MQTT. In that conversion the binary payload gets converted to a Json payload.
Kiwi
-
Thanks for your post, and I agree with MQTT as a hub, but think pushing it down to the RFM69 layer is not advisable, especially since one of the main use cases is battery operation, where every byte transmitted costs precious milliamps. I have gone for a very small binary structure, with a Mote with ethernet shield acting as a gateway to MQTT. In that conversion the binary payload gets converted to a Json payload.
Kiwi
Thanks Kiwi. I absolutely agree that a mandate must be to minimize RF data transmission in sending/receiving meaningful messages to devices. Indeed, we can all design our own rf protocols and pack what we want/need into it very tightly, but, again that takes us away from interoperability. If our common objective, as an industry, is to build and deploy devices that can all inter operate together using common protocols and methods, even across rf networks, then I think MQTT out to the device is also necessary.
And, I believe that the MQTT protocol can be deployed in such a way as to also keep data transmission to a minimum as well. For example, in the snmp protocol, devices, objects, and variables are all represented by a byte or two each separated by periods when messages are transmitted. Those numbers are translated to human readable notation by the management/presentation software translating a "mib" file which defines what the device is and what the individual parameters are and mean. In my work so far, I believe its possible to manage and control a device, which has any number of variables which can be read or set, with MQTT messages, in a message payload of of around 40 bytes. or less. That's pretty efficient and low power RF friendly in my opinion.
When a MQTT message is received by the RF to IP gateway device, it appends on a few more sub topics on the front of the MQTT message for identification and management purposes, then sends that on to the MQTT broker. All done.
Now, the presentation/management system, which subscribes to the relevant MQTT topics on the broker, takes those "/ encoded" MQTT messages and translates them into something human readable. This opens up the possibility for an endless number of management station designs using any number of presentation methods. Json may well be a very likely candidate for that. It could be anything but the point is that all the devices out there are all using the same MQTT messaging format so that maybe, someday, we can achieve interoperability!
-
For battery operated devices and memory constrained you can use mqtt-sn protocol. Works with sleepy devices.
Documentation on both, MQTT & MQTT-SN: http://mqtt.org/documentation
Open source projects:
https://github.com/X13home
https://github.com/ty4tw/MQTT-SN
https://github.com/boriz/MQTT-SN-Arduino
https://github.com/bittailor/BtMqttSn
-
For battery operated devices and memory constrained you can use mqtt-sn protocol. Works with sleepy devices.
Documentation on both, MQTT & MQTT-SN: http://mqtt.org/documentation
Open source projects:
https://github.com/X13home
https://github.com/ty4tw/MQTT-SN
https://github.com/boriz/MQTT-SN-Arduino
https://github.com/bittailor/BtMqttSn
Thanks for the tip and links! I'll look into it (now that my 'service' provider decided to restore service to our house! - been out since Sunday night and it only took 4 calls and two tech visits to fix...).
Tom
-
Thanks ricka...
I've been watching the MQTT-SN development as well. The way I look at it is that, yes, it's pushing the MQTT protocol out to the node, but the architecture still requires a gateway to forward the messages on to the broker on the IP network. An RF to IP gateway will be required however you try to do this. Implementing the MQTT-SN library into each node requires additional code overhead, memory, and resources. As well, it requires that each node be pre-programmed (hard coded) with very specific variables to talk with specific gateways and mqtt message topics.
I don't think it's necessary to put an MQTT library on the devices. In my mind and opinion, all the devices have to do is transmit their data formatted into a predefined MQTT topic format. The device doesn't even need to know anything about MQTT or MQTT messages, it simply has to format it's output strings/messages into a format that is MQTT friendly and send it to the gateway. No more overhead then how everyone currently formats their data in one way or another as a string of data separated by colons or whatever. The "RF to IP" gateway takes care of understanding how to talk to an MQTT broker. With this concept, and in my rough design so far, any given device only needs to be hard coded with a manufacturer id, model number, and unique identifier (say a mac address), and a default network id (say 254) to start up. From their, the gateway, through a dhcp type exchange, can assign it a new network id and now knows it exists and what it is and what mqtt topics to get and send to it for management.
In my mind, to achieve interoperability, any given device, designed to work on a "moteino" rf network should be able to just plug in a go...lol..in my mind. Let's see if we can get there ;)
-
MQTT-SN protocol specification allows you don't implement all features. You can implement only required features:
6.8 PUBLISH with QoS Level -1
This feature is defined for very simple client implementations which do not support any other features except
this one. There is no connection setup nor tear down, no registration nor subscription. The client just sends its
PUBLISH messages to a GW (whose address is known a-priori by the client) and forgets them. It does not care
whether the GW address is correct, whether the GW is alive, or whether the messages arrive at the GW.
MQTT-SN protocol specification is a good roadmap how to implement your protocol with minimal efforts and wheel reinvention. For RF protocol you can use Felix's library, jeelabs or radiohead library. It will be interoperability issues with devices using different software libraries/RF protocols, RF chips, RF modulations and etc.
-
MQTT-SN protocol specification allows you don't implement all features. You can implement only required features:
6.8 PUBLISH with QoS Level -1
This feature is defined for very simple client implementations which do not support any other features except
this one. There is no connection setup nor tear down, no registration nor subscription. The client just sends its
PUBLISH messages to a GW (whose address is known a-priori by the client) and forgets them. It does not care
whether the GW address is correct, whether the GW is alive, or whether the messages arrive at the GW.
<snip...>
Gee, those guys must have been looking over my shoulder when they came up with this one... ;)
Still have to read the specs. It should be an interesting learning time, in any case...
Tom