I played around with this nut/bolt thingy on the backside of the upper jockey wheel and it seems to work much better. Apologies for my lack of technical jargon.