Two weeks isn't very long to be expecting conditioning results. I would think closer to two months would be more realistic.

As far as your performance, maybe riding the geared ride once every two weeks is a little too intermittent, eh?