After a long time being a reader of this forum, I finally took the leap in registering.
Like some of you, I'm also building a remote sensor network. In my case, it is to monitor beehives.
My question is towards the design of the gateway. I will have quite some nodes (starting at 10 or so but it could go up to over 100 or more) and am wondering what's the best hardhare and design to use for the gateway. For the node wifi, I picked the rfm69w.
I have some arduino experience but no Rpi experience yet. Not sure if it's a lot harder to work with.
What's your opinion? Any write-ups you guys know of?
Assuming you will want to relay (pass on) the sensor data to a more powerful processing gateway (host computer of some kind ...), then RaspberryPi or similar small computing open source platforms are ideal. However if you've never touched linux it will be quite a learning curve. Imagine Arduino learning curve but squared or cubed.
Fortunately by now the internet is full of tutorials and free info on how to setup your Pi, how to get started with pretty much anything you need it to do. I have my own writeup to set it up as a gateway between the sensor network and an internet connected (and secured) socket server which can display real time interaction between your smartphone or any other device and your sensor network. Again, easier said than done, but the frustration slowly turns into fun when it starts working ;)
If your beehives are very remote somewhere and you need almost-realtime interaction then your best solution would be a GPRS equipped gateway. Obviously that needs cellular coverage, and still will be slower than broadband.
Thanks for the reply.
The gateway will be an intermediary. All data will be pushed immediately (as it comes in) to a web server. I am sure that the Pi could do it but I'm not sure if it's worth the overhead. Basically the gateway will have an ethernet connection (wired) at all times to a router.
I understand that, when using a Pi as a gateway, you connect a Moteino to the Pi and get the readings this way. How would I do it when using an Arduino board? Should I use a RFM69W shield (not sure this exists) or just use the Moteino itself as a gateway.
The connection to the router is easy when using an Arduino through an ethernet shield. Does anybody know of a Moteino that has been connected through an ethernet port to a router?
Quote from: Quobu on January 02, 2014, 12:19:11 PM
I understand that, when using a Pi as a gateway, you connect a Moteino to the Pi and get the readings this way. How would I do it when using an Arduino board? Should I use a RFM69W shield (not sure this exists) or just use the Moteino itself as a gateway.
I have a hard time with this and have to keep reminding myself of the fact that a Moteino basically
IS an Arduino. Smaller form factor, 3.3v and requires an FTDI to talk (unless using the USB Moteino), but basically an Arduino. Note that when you program it you select Arduino Uno as the target board. (Praise be to Felix! I don't know if I would have had the patience to deal with screwing with settings on the IDE again after my previous experience trying to use the MIT HiLowTech writeup to program AtTiny85's.)
Quote from: Quobu on January 02, 2014, 12:19:11 PM
The connection to the router is easy when using an Arduino through an ethernet shield. Does anybody know of a Moteino that has been connected through an ethernet port to a router?
[Barney Fife voice]
Well, yeah, of course the connection to the router is easy when using an Arduino through an ethernet shield. I'm just making sure YOU know how to do it!
[/Barney Fife voice] :-[
I'm starting small, but that is on my list (towards the end) of stuff to do, hopefully building knowledge on the way. If you have code to do this, please post it. So many of the other places I have seen code to do Ethernet connect are way over my head.
It looks to me like there could be some real value to designing either:
- A Moteino (or Moteino USB) on a PCB that matches the Arduino footprint - thus allowing use of standard shields.
-OR-
- An interposer card / shield that a Moteino could plug into that would accept standard shields.
Either one would probably require level shifters to make the Moteino 5v tolerant on the inputs.
Both are well beyond my abilities at this moment, since I have -ZERO- experience with either KiCAD or Eagle, so don't wait on me to do it.
For the record, I'm not begging anyone to do it for me either, least of all Felix. (I want some of those switch Motes!) Just putting it out there as an idea to see if anyone else thinks it would be a good idea, or if it is totally unnecessary because ____________.
LG
Looks like the Ethernet shield uses SPI to talk to the connected Arduino. The Moteino has a SPI bus it uses to talk to the radio module (and the SPI Flash, if installed). You should be able to connect wires from the Moteino to the Ethernet shield, 3 for the SPI bus and 1 GPIO for select (SS). I would guess the Moteino could handle adding the Ethernet shield to its SPI bus, but I'm not sure, it might be too much for it.
Another option is a serial to Ethernet device like this - https://www.sparkfun.com/products/9476
Just setup the device for your network and connect to the Moteino serial port (TX, RX, ground).
If you connect your gateway Moteino via serial to a Raspberry Pi, Beagle Bone Black (or something else running linux), you can just use socat to "connect" the serial and Ethernet ports together - http://stackoverflow.com/questions/16538160/raspberry-pi-how-to-send-serial-command-using-ethernet
To keep it straight forward and not having to learn the Pi, I think I'll try to stick with the arduino's.
After a bit of research, it seems like using this one is an option:
https://www.sparkfun.com/products/9473
since it also uses the W5100 chip, which is also used on the official arduino breakout board.
Now the question is if the SPI interface can handle this board besides the radio communication. For my specific case, I can do without the SPI flash (although, I'd rather have it) if this means I can use the Wiznet module.
Any thoughts of running these 2 (or 3) devices via SPI?
I would probably advise for going with a Pi, I think it makes most sense. Gives you most flexibility, power, security etc at still a very low price point. I think the learning curve is still there for an Ethernet shield, maybe not as steep but still. What I typically do with the Pi when I need certain options is configure everything the way I want it then save an image (or image checkpoints with certain sets of software installed) and reuse later if needed, that way I don't have to reconfigure everything.
Also, the Moteino is just an Arduino. It could probably handle a wiznet5100 but when you have an RF transceiver with lots of potential inbound traffic, you want to spend as much time as possible listening. My RFM69 library is interrupt based with polling for received packets. IOW you put the transceiver in RX mode, when a packet is received an interrupt takes care of reading the packet into a local buffer, then you have to poll that buffer using the library functions to see if a packet is ready. If you do lots of other stuff with the ethernet, then you might get packets overwritten, and hence lost sensor data. But that might not happen depending on how much traffic there is ... it's all a tradeoff. It's hard to guess what's the best solution, but it's always better to let one component do one thing, ie let Moteino handle the RF data alone, then let a Pi or other computer handle the ethernet and web stuff.
I 2nd Felix, connecting the Moteino to another device (Raspberry Pi, BBB, PC, etc.) to handle Ethernet is the way to go.
Connecting an Ethernet shield to the Moteino may work, but if you run into problems you'll likely spend a lot of time tweaking and may have to abandon this approach altogether.
I feel a Raspberry Pi is safer because you've got a lot more options with a full linux device - if one approach doesn't work, you've got many other things you can try on linux. Let's say simply forwarding the serial data from the Moteino gateway to a host via socat doesn't work, you can try a shell script or Python program, etc.
Check out this project that's similar to what you want to do - http://www.springamp.com/20131027/wireless-sensor-motetino
Oh and I forgot to mention - I don't even want to think of what it will take every time I want to change the way I want to post the data in a different way or in a different format over HTTP or over UDP/TCP etc. You will sit there many nights tearing your eyes out trying to guess why the arduino+ethernet shield doesn't do what you want. I think that time investment would be way more efficient in getting to know a computer like the Pi.... just my extra 2 cents...
If it is of any interest guys, I am currently working on an Arduino shield that provides the following support:-
- Either Adafruit CC3000 BOB or GB WizFi210 BOB Support
- RFM6xxModule Support
- BLE112 Module Support
- 4D systems uLCD-xxPTU support
It will eventually be released as OS via my wwwGorillaBuilderz.com (http://wwwgorillabuilderz.com) site
If you want more information pls pm me.
Regards,
Dale
Dale - Are your products focused more around the standard Arduino shield layout?
Hi Felix. Yes they are based on the standard shield footprint.
From personal experience with the Arduino Ethernet, while it is a solid, stable device, you very quickly run up against its limits. For me the tipping was HTTPS/SSL communications to/from my home network. None of the Arduino solutions (Ethernet, WiFi, Adafruit CC3000, etc) can do HTTPS/SSL. (The Yun can do SSL, but the Yun is similar to a Moteino connected to an RPi via serial.) I also have no clue how hardened these devices are to malicious packets coming in from the outside and I was opening ports like crazy on my firewall (port forwarding) for outside the home access.
My initial solution was to try to put the Arduinos behind an nginx reverse proxy to terminate the SSL at my firewall and send clear text to/from my arduinos, but I could not get a stable, reliable system. I was able to use an nginx reverse proxy with owncloud and other webservices hosted in my home, so the problem had to be with the Arduino ethernet. The next step was to have nginx on an RPi do all the heavy lifting (similar to Felix's GarageMote)--terminate SSL, provide HTTP(S) responses, etc on the RPi and have the RPi send, via serial or ethernet, simple commands and queries to the moteinos/arduinos as needed. I was immediately rewarded with a stable platform.
I now wonder why I wasted so much time trying to put a square peg in a round hole, so to speak: Let the RPi do what it is good at and the Moteinos do what they are good at.
Quote from: priority on January 03, 2014, 01:27:23 AM
From personal experience with the Arduino Ethernet, while it is a solid, stable device, you very quickly run up against its limits.
. . .
I now wonder why I wasted so much time trying to put a square peg in a round hole, so to speak: Let the RPi do what it is good at and the Moteinos do what they are good at.
That was my exact thought after playing with an Ethernet shield and during the development of the Moteino idea. It seemed so pointless and such a waste to make every node either wifi or ethernet based, or actually use an ethernet shielded gateway with an Arduino chip as the brains. Arduinos are great to do simple dedicated things, and Pi computers are great for doing heavier work that needs to be much more abstracted, while still very powerful in terms of computing. If you factor in the price of a Pi and the small size, it's a no brainer ...
Thanks everybody for all the replies. You have me convinced. I'll be using the Raspberry Pi or the BeagleBone Black for the gateway. Not sure which one yet I'll be using. They both seem to have their advantages and disadvantages.
Quote from: Felix on January 02, 2014, 05:09:47 PM
My RFM69 library is interrupt based with polling for received packets. IOW you put the transceiver in RX mode, when a packet is received an interrupt takes care of reading the packet into a local buffer, then you have to poll that buffer using the library functions to see if a packet is ready. If you do lots of other stuff with the ethernet, then you might get packets overwritten, and hence lost sensor data. But that might not happen depending on how much traffic there is ... it's all a tradeoff.
This is kind of related, but I'm curious if anybody has any stats on running networks of Moteino's and how much data one node can handle. My setup (which I'm guessing is fairly common) is 1 Gateway node being yelled at by a bunch of Moteino's.
How many incoming messages can a single Moteino reasonably be expected to handle?
It can handle a lot. Basically each node "listens" before it sends to make sure there's no other RF traffic or noise, this is to avoid collisions So if there is a lot of traffic, there might be slight delays (on the order of tens of ms at most). The sendWithRetry() function can be used to retry a send until it gets an ACK, so really you should get a message through with a very high percentage of success.
@Quobu,
If you have a look at the following link this should answer your question, does a Hope RFM69W/HW Arduino shield exist?
You should find a couple of photos and the Eagle schematic and pcb files.
http://jeelabs.net/boards/6/topics/2934?page=2
cheers,
Quote from: Felix on January 10, 2014, 09:03:04 PM
Basically each node "listens" before it sends to make sure there's no other RF traffic or noise, this is to avoid collisions
Thanks Felix.
If I'm reading this right that's a feature of the RFM69 library (Specifically the canSend function), not something built into the hardware right?
Quote from: schulz on January 13, 2014, 04:22:55 PM
If I'm reading this right that's a feature of the RFM69 library (Specifically the canSend function), not something built into the hardware right?
Correct.
Another perspective.
While I make no claims that my gateway is in any way a "best" design it has been working reliably for just over a year. I started with JeeNodes and have only recently discovered Moteino. All of my currently functioning nodes are JeeNodes with a 915MHz RFM12B radio. My gateway is a JeeNode with the ENC28J60 Ethernet shield. I have about 10 remote nodes at this time. When the gateway receives a packet from a remote node it checks the node and data index combination against a reference list. If there is a match it sends the data to my Linux webserver by way of a php page. That page takes the input from the JeeNode gateway (ID and data) and stores it in a MySQL database. To prevent filling the database with useless duplicate data I only store the data if it has changed by more than a configurable delta. It also gets stored ~once/hour even if it has not changed. If no data has been stored in the last hour I get a text message so I can respond. My data is only important to me so I choose to limit the text message to my normal waking hours! I have had to reset the gateway a couple times.
I have some php based web pages using RGraph to display the saved data.
I'm sure the Raspberry Pi adds capability but it clearly is not necessary. It is great to have so many choices...
John
I will be using the ChipKit WF32 as a gateway. I already ported the RFM12B library to it, and will be porting the RFM69W.
The WF32 is a PIC32MX Arduino compatible board with 512K Flash, 128K SRAM, WiFi module, microSD and costs only $69. You can get an RFM12B or RFM69 module for about $7 or $8.
I thought about using a Moteino connected to a microcontroller, but that would add another layer of software, which could be avoided by connecting the RF module directly to the gateway micontroller.
By using a board that is Arduino compatible, I can use some of the same code on the Moteino as on the gateway - so less code to maintain. The gateway is using JSON messages to communicate with the server. I'm using the aJson library for that, https://github.com/interactive-matter/aJson. It makes it easy to parse and format JSON messages.
Read more about the RFM12B library here:
http://lowpowerlab.com/forum/index.php/topic,283.0.html
If you're interested in sqlite/python backend and jquery/jquerymobile/html5 frontend, let me know. I have libraries I'm about to release in with a GPL-compatible license that will save you some time. They'll be attached to a commercial gateway product that will be continually developed and supported. Should be an easy port to MySQL or other dbs if you have a favored flavor.
Colin
ColinR, I am interested in your libraries. Can you post a link?
Quote from: BeagleNode on January 15, 2014, 09:00:17 AM
Another perspective.
While I make no claims that my gateway is in any way a "best" design it has been working reliably for just over a year. I started with JeeNodes and have only recently discovered Moteino. All of my currently functioning nodes are JeeNodes with a 915MHz RFM12B radio. My gateway is a JeeNode with the ENC28J60 Ethernet shield. I have about 10 remote nodes at this time. When the gateway receives a packet from a remote node it checks the node and data index combination against a reference list. If there is a match it sends the data to my Linux webserver by way of a php page. That page takes the input from the JeeNode gateway (ID and data) and stores it in a MySQL database. To prevent filling the database with useless duplicate data I only store the data if it has changed by more than a configurable delta. It also gets stored ~once/hour even if it has not changed. If no data has been stored in the last hour I get a text message so I can respond. My data is only important to me so I choose to limit the text message to my normal waking hours! I have had to reset the gateway a couple times.
I have some php based web pages using RGraph to display the saved data.
I'm sure the Raspberry Pi adds capability but it clearly is not necessary. It is great to have so many choices...
John
I also wanted similar solution but was not sure whether ENC28J60 would work with Moteino. I only need to send HTTP GET request to my website which can extract the values of RF signal and would be able to store the values in MySQL.
I think having additional device may not not required if I can send the signal to website with just 5$ ENC28J60.
I just ordered 3 moteino and would try to test with ENC28J60 and if it works then it would very powerful and cheap solution for my requirements.