When you have the LDO, and also the radio/flash, you have to account for those components.
They have VCC limits and a dropout for the LDO.
Also, the supercaps take a "long" time to charge from a solar cell when depleted. So having the Moteino/load attached during charging from depletion will not work well.
Either you charge the supercap from a PSU, then connect it to the solar cell and Moteino so the cell can sustain it, or charge it fully from the cell and then attach the Moteino.