Moteino & W5100 ethernet SPI support / SPI_HAS_TRANSACTION

[Robert]

Hello Felix
I have tested the RFM_SPI_TRANSACTION_attemp library which is actually not working (RFM is hanging with an Ethernet controller).
It is quite similar to the Kiwi version, however  one piece is missing (see modification 3.)
Modification 1 and 2 are related to compilation errors.
 
Modification: RFM69.h
1. Suppress “SPISettings _settings;”  which cause a compilation error for incorrect type

#ifndef SPI_HAS_TRANSACTION
    uint8_t _SREG; // Saves the Interrupt state before disabling
// !!!! ROB    SPISettings _settings;
#endif

Modifications in RFM69.cpp
1. Suppress after the association of SPISettings to _setting after SPI.begin ();
  SPI.begin();
/* !!!!!!! ROB
#ifdef SPI_HAS_TRANSACTION
  _settings = SPISettings(4000000, MSBFIRST, SPI_MODE0);
#endif
*/

2. replace _settings by its actual function in RFM69::select ();
void RFM69::select() {
#ifdef SPI_HAS_TRANSACTION
  #if defined (SPCR) && defined (SPSR)
    _SPCR = SPCR;
    _SPSR = SPSR;
  #endif
// !!!!! ROB  SPI.beginTransaction(_settings); // Replaced by …
  SPI.beginTransaction(SPISettings(4000000, MSBFIRST, SPI_MODE0));// !!!!! ROB
#else
…..

3. Most Important add SPI.usingInterrupt(_interruptNum) in the bool RFM69::initialize(uint8_t freqBand, uint8_t nodeID, uint8_t networkID)
……..
  while (((readReg(REG_IRQFLAGS1) & RF_IRQFLAGS1_MODEREADY) == 0x00) && millis()-start < timeout); // wait for ModeReady
  if (millis()-start >= timeout)
    return false;
//!!!! ROB
#ifdef SPI_HAS_TRANSACTION
  SPI.usingInterrupt(_interruptNum);
#endif
//!!!!!   
  attachInterrupt(_interruptNum, RFM69::isr0, RISING);

  selfPointer = this;
  _address = nodeID;
  return true;
}

[TomWS]

3. Most Important add SPI.usingInterrupt(_interruptNum) in the bool RFM69::initialize(uint8_t freqBand, uint8_t nodeID, uint8_t networkID)
……..
  while (((readReg(REG_IRQFLAGS1) & RF_IRQFLAGS1_MODEREADY) == 0x00) && millis()-start < timeout); // wait for ModeReady
  if (millis()-start >= timeout)
    return false;
//!!!! ROB
#ifdef SPI_HAS_TRANSACTION
  SPI.usingInterrupt(_interruptNum);
#endif
//!!!!!   
  attachInterrupt(_interruptNum, RFM69::isr0, RISING);

  selfPointer = this;
  _address = nodeID;
  return true;
}

THAT is sort of key to the whole thing working, isn't it!!!   

Good job.
Tom

[Felix]

Thanks Robert, this is great, I will have to try to test and merge these changes when I get a chance.
So did you start off with the RFM69-SPITransaction-attempt branch and just added the changes you mentioned?

[Robert]

Felix,

Yes I just did those changes to the RFM69-SPITransaction-attempt branch, and I am running my Arduino Ethernet gateway with this version.

Now I have two remarks.
1. The RFM69-SPITransaction-attempt library is slightly different that the "last official" RFM69 one (so I will check the patch with the last version, and give you a feedback).
    Note that I don't use the ATC option, but I will also try it with the updated version
2. The interrupt patch uses direct access to AVR SPI registers, which is an issue while using this with ESP8266 modules.

As far I have test it the current RFM69 library is compatible with the ESP8266 SPI library and the interrupt issue of the Arduino Ethernet shield does't occur with the WiFi interface, so the patch is not necessary for ESP8266.
Introducing direct access to AVR registers or SPI functions that are not defined in the ESP8266 SPI library makes the RFM69 library incompatible with the ESP8266 which is annoying because two different versions have to be maintained.

This requires more work to find if compatible solution may be found, but I will give a try !

Robert

[Robert]

Voil� 
After spending some time to read all the comments regarding the SPI library interrupt issue and improvement, let me summarise my findings expecting to clarify this topic.

Motivations:
ESP micro-controllers allow concurrent usage of Ethernet Wireless and RFM69 Wireless modules and is a cheap alternative to Arduino WiFi Shield to build Ethernet/RFM Gateways.
The RFM69 library is compatible with the ESP8266 devices such as WeMos, NodeMCU, HUZZA, etc… more precisely the ESP8266 SPI library (currently 2.3.0).
See https://lowpowerlab.com/forum/projects/(update)-rfm69-library-for-esp8266-moteino-compatible!/75/
(Beside possible SPI clock speed adjustment; see https://bitbucket.org/xoseperez/rfm69gw), and below: SPI depreciated functions).

For Ethernet wired configuration, using RFM69 transceivers and Arduino Ethernet board based on WizNet5100 (Arduino Ethernet or Arduino Ethernet Shield) there are SPI interrupt conflicts that are hanging sketches; this issue is solved by a new version of the SPI library and RFM69 library patch (currently not implemented).

Goal:
To have a RFM69 library operable in both Arduino Ethernet and ESP8266 environments.

Current issues:
Arduino AVR environment.
When using SPI interrupts (case of W5100 and RFM69), interrupts conflicts may occurs.
This problem is described by http://dorkbotpdx.org/blog/paul/spi_transactions_in_arduino
The workaround is a new SPI library (since Arduino IDE 1.5.8 )
This new library implements 3 new functions while accessing (select / unselect SPI):
 From synchronous access:
  SPI.beginTransaction(SPISettings);
  SPI.endTransaction();
 For asynchronous access:
  SPI.usingInterrupt(number);
Which are well documented, see http://www.pjrc.com/teensy/td_libs_SPI.html

To activate conditionally these new functions, for compatibility with older IDE compilers, a statement SPI_HAS_TRANSACTION is defined in the new SPI library.

A patch using these functions for the RFM69 library was submitted by kiwisincebirth (see https://github.com/kiwisincebirth/RFM69) but was never actually implemented.

So the first fix is to ensure that this patch is working in an environment where an RFM transceiver is activated together with an Ethernet shield.

ESP environment
If the SPI_HAS_TRANSACTION i.e: SPI.beginTransaction(SPISettings) and SPI.endTransaction() is implemented in the ESP8266 SPI library (2.3.0), the (SPI.usingInterrupt(number) is not, which is a first issue.
Also, the kiwisincebirth  patch makes access to an AVR register: SREG (Status Register) this register doesn’t exit as such in the ESP8266 which is a second issue.
Moreover the kiwi patch which is working for Arduino AVR, was removed for an obscure reason to fit the ESP environment see https://github.com/esp8266/Arduino/issues/1943 and https://github.com/LowPowerLab/RFM69/commit/ec90680e008c366dc7340ec1aada3942f8592707

So the second fix is to ensure that the patched RFM69 library is still working with the ESP8266 SPI library.

Note:
The RFM69 library uses references to other registers such as SPCR (SPI Control register) and SPSR (SPI Status Register). However conditional access to  these registers is made (typically  via : if defined (SPCR) && defined (SPSR)). Because SPCR and SPSR are not defined in the ESP8266 library, this is not causing a problem.

Current approach
With this approach the new SPI functions will be activated according to the SPI library version using “#ifdef SPI_HAS_TRANSACTION”
Saving and restoring the SREG is more likely unnecessary (was verified during tests), but for backward compatibility, reference to this register is made conditional for AVR environment.

Now, to avoid activating the function SPI.usingInterrupt(number), an extra test is done to one the processor type (typically ESP8266) using “#if defined (SPI_HAS_TRANSACTION) && !defined (ESP8266)" before activating this function.

Depreciated SPI functions in RFM69 Library
Small library improvements may be applied.
1. Explicit reference to Interrupt Number, which could be replaced by: digitalPinToInterrupt(interrupt_pin)
2. SPI functions such as:
        SPI.setDataMode();
        SPI.setBitOrder();
        SPI.setClockDivider();
   Are replaced by SPISettings(speed, bit_order, mode) see https://www.arduino.cc/en/Reference/SPISettings (which by the way is also implemented in the ESP library).
 This feature allows to use the appropriate speed for a SPI device independent of the processor clock

Conclusions:
The RFM69 library issues described above are not related to a MOTEINO environment but related to the usage of this library with SPI Ethernet controller or ESP controllers.

One has to decide if this library update is to be implemented or not in the frame of MOTEINO.

My point of view is that it should be implemented to make usage of the new Arduino SPI library functions (MOTEINO uses AVR processors), despite the fact that the ESP SPI library is not implementing the SPI.usingInterrupt(number) function (or a similar dummy function) see http://www.esp8266.com/viewtopic.php?f=28&t=11998&sid=59d9547d624a26a1de38a1e6ebbf3932#p56001f=28&t=11998&sid=59d9547d624a26a1de38a1e6ebbf3932

In this case the only solution for people wanting to use the RFM69 library for ESP micro-controller is to adapt the library avoiding to activate SPI.usingInterrupt(number) controlling the environment using “if defined (ESP8266) statement”. 
Tests
I did tests with an Arduino ATMEGA2560, Arduino Ethernet Shield and  RFM69HW transceiver as network coordinator (sending converted RFM messages via  Ethernet MQTT to a Raspberry OpenHab broker) for my Home automation MOTEINO RFM69 nodes . This version is currently running in my operational environment, without problems.

I have also loaded  an  ESP-12 with a similar gateway sketch, I use the Struct_send.ino sketch from the RFM69 library to send RFM data that are forwarded via Wifi to my Raspberry Broker without problems.

I plan to replace my coordinator by WeMOs module on a Charles-Henry Hallard WeMos-RFM gateway, I will keep you inform if I encounter new problems.


For the record I have posted a RFM69.cpp and RFM69.h updated library on https://github.com/rrobinet/RFM69_Libary where my modifications are clearly indicated (!!! ROB)

NB: RFM_ATC library was not tested because of compilation errors, but this is another story (I must admit that I did not spend too much time to debug that one)
 

[Robert]

For information the RFM69 library on WeMos is also working
Robert

[JP]

Hello,

We are trying to develop a RFM69HW gateway with a W5100 chip. After reading all the posts of the topic...

I have downloaded the new version of the RFM69HW of Robert

And I am not sure if it is necessary, or finally not, to install the modified Ethernet Library with the CSPIN argument.

I am also confused if it is, finally, necessary or not to add the cli() and sei() lines into the w5100 library.

Thanks for your work, it is really awesome.

Congratulations!!!

[Robert]

Hi JP,

I do not use the W5100 directly, what I did use is the Arduino Ethernet Shield with an ATMEGA2560 and the Ethernet Library, (which is using the W5100 one) together with an home made RFM69HW shield.
As I wrote previously, the issue is while multiple SPI slaves are used (RFM69 and W5100 for instance)

This issue is solved by the new (current) SPI library, that introduces a way to control / co-ordinate SPI transactions. The SPI.beginTransaction() at the time a specific SPI device is selected and the SPI.endTransaction() at the end of the selection. These SPI library functions are taking care of the interrupts.
Now the W5100 library is probably not updated with these new functions, however using these ones with the RFM69 library looks to be sufficient.
So when using an RFM69 and a W5100, you should use my patched RFM69 library, and you should NOT enable and disable interrupts with cli() and sei() in a modified Ethernet library such as the one described by http://harizanov.com/2012/04/rfm12b-and-arduino-ethernet-with-wiznet5100-chip/
You may perhaps modify the W5100 library with the SPI transaction functions, but this is another story

However while connecting a RFM69 transceiver on an Arduino, you should take care of a correct configuration of the SPI signals (I/O pins) and use an appropriate Slave Select pin.

Note: If you don’t need several I/Os, I strongly recommend you to have a look at the WeMos module and at the WeMos gateway of Charles-Henry Hallard (ESP8266 WeMos Shield for HopeRF RFM69 and RFM12 Module see https://github.com/hallard/WeMos-RFM69). Which with a little effort .... will allow you to build a powerful and cheap Ethernet (WiFi) (WEB) gateway.

Robert