Just as I thought, good work Luka 
So bottom line: need 12VAC+, and replace 100uF with 10uF.
Nobody else reported problems but thanks for reporting all this and working towards resolution and making it better for everyone.
Was going to say that, the current gain of the optoisolator is 20%, which means with 8VAC the PNP transistor will only be on for a proportion of the 50Hz waveform, and the RC constant of the filter cap with the 1k5 resistor and LED is too short to hold the voltage up for a complete cycle. Increasing the capacitor to increase the RC constant should do it, as would probably increasing the 10k pull-up resistor on the PNP transistor. I'm not sure you need 12VAC with a pull-up increase and a 100uF cap, unless that's not the only problem with 8VAC.
Mark.
