1284p SRAM vs 328p&EEprom buffering on a data logger

[EdM]

I'm finally getting some preliminary test results in from my first batch of Moteino Mega based data logger builds:

http://edwardmallon.wordpress.com/2015/06/28/testing-power-use-with-a-complex-duty-cycle/

I was quite surprised to see that test come to a draw, so now I am wondering if there is a way I can make the Moteinos draw less power (/run slower?) during mcu up time. I already use the Rocket scream lowpower lib with 1284p support for sleeping.


And: some of the Moteino Mega builds are extremely choosy about which SD cards they will work with.  Though the Megas I used have no flash, and no transceivers, SD cards that go into sleep immediately on a typical Ultra based logger take a several seconds to go into sleep mode on the Moteino Mega based builds. It could just be a soldering error on my part, but after being unable to fix the problem a few times now, I am left wondering if something else might be putting other traffic on the SPI lines which is preventing the SD cards from immediately going into sleep mode...?

[Felix]

Interesting project!

I wonder if you really need the SD cards? Maybe could you get by with a large-ish FLASH chip instead? The Moteinos come standard with the 4mbit FLASH MEM chip [optional]. But there are mcuh larger chips available with the same pinout/instruction set and should be SPIFlash lib compatible. That would offer more logging space and they are quite low power and easy to deal with.

[EdM]

I think I will eventually reach that point when I get better at programming, but for now I want to see how far I can get with the SD cards in place.  Tethered downloads can be a pain for fieldwork when you have 20+ loggers in a cave, and short of working on a portable download device, SD cards are the simplest option for fast turnaround on site.

I keep hearing rumors of special low current cards from TwinMos or other interesting vendors like ( http://www.globalforte.com/download00034678/Spyrus/Products/Rosetta_SD_MiniSD_MicroSD_Series_II.pdf )
but I have yet to get my hands on them for testing.  I would also love to hear about it if anyone has managed to successfully do multiple initializations with SDfat (or some other SD library), as that would allow me to pull power from the sd cards rather than just letting them sleep.

[Felix]

How much does the SD card pull when idle?
If you cut power off to it, you should be able to re-initialize when you power it back up. Depending how the FAT libs are implemented you might need to do some garbage collection in between.

[EdM]

Such a simple question, but it has taken me months to work through all the 'related issues' (with much appreciated guidance from Mr. Greiman over at the arduino.cc forum!).

If you pull up the unused data lines, and you don't have a counterfeit sd card, and the card is not too old, and the phase of the moon is right...etc. I call an SD card good if it sleeps around 60-70 uA immediately after the mcu goes to sleep. 

Many cards take some time to think about whether they want to go to sleep or not after the SPI clock is shut down, and some cards are sneaky, going to sleep right away and then slowly creeping back up to 1-2mA after a the logger has been asleep for a couple of minutes. 

The "time to think about it" is what has been affecting some my Moteino Mega builds though I am using well tested cards that sleep immediately on the Ultras.  I have had to dig into my reserve of "the very best" SD cards (generally old Sandisk 128mb cards) to get those units into decent sleep current territory.  Some of those older Sandisk cards are bulletproof, and sleep well no matter what I do to them.

And for anyone who is really interested, Luke Miller has had similar SD card sleep issues over at his wave height project:
http://lukemiller.org/index.php/2014/11/revised-open-wave-height-logger-battery-tests/
though his loggers run at very high sample rates compared to mine...

[Felix]

So it sounds like there is much variability in the SD cards' behavior and it's a hit/miss to get them to sleep as expected.
Unless you can find a good source of known SD cards that perform deterministically, it's hard to scale such a system without a lot of wasted time and resources.
Another Q is how much data (in bytes) do you expect to log over a given period of time (1 month?)?

[EdM]

I record alot of other data like power supply voltage, RTC coin cell voltage, post SD writing supply, etc. Simple one-sensor loggers generally store about 150,000 bytes/characters in CSV files on the SD per month. Multi sensor units can be 2-4x that depending on how many are attached.  Raw integer versions of that data would probably be 1/2 to 1/3 the size.

Generally the installations take a great deal of time and effort to get to, so I am hoping for units that will run for at least 12 months before they need servicing.

WRT re-initialisation & cleanup. I am not a programmer (and my electronics background is pretty modest) so I don't think I would be able to tackle low level hacks to help those libs handle multiple starts if they don't support it natively.

There are also short interval & sensor initialization issues that make shutting down the entire Arduino less helpful to the power budget in some situations. That's why I have been cautious about going down the route that some others have tried:

https://www.tindie.com/products/Dead_Bug_Prototypes/extreme-low-power-data-logging-shield-for-arduino/

[Felix]

12months is a long cycle.
I would suggest taking a look at larger FLASH MEM chips. It sounds like the default 4MBIT (512Kb) might just be sufficient but perhaps a 16MBIT (2Mb) might cover all your cases.
The SPIFlash lib probably has everything you need to read/write a stream of bytes (perhaps a struct or whatever format you use for logging) to the chip. Unless I'm missing something, this could really simplify your project too. The cost would go way down, the chips would be very low power and can be slept for even lower, also the space savings and all the soldering involved would be an instant gratification.

[EdM]

I used I2C eeproms in the first case because they were directly code compatible with the memory on RTC breakout I was already using. So all I had to do was change the address in my existing scripts. But I've gone through my first batch of Megas already, so I will add memory chips to the next ones so I can do some testing. 500K is a lot to work with.