You just need to get used to it. I'm roughly 125lbs and ride 50mm front and back with aero spokes most of the time. First few times I got caught in the wind it scared the **** out of me, but now it's no problem to keep straight in the crosswinds. I have a rear disc cover I use for TTs and tris as well, and it's even more stable with that in the wind.