If you put on enough miles using proper technique, before you start hammering big gears, you should be fine(although I'm not sure that skidding is advisable).
I ride a 46x17 or a 46x20 depending on the ride that day FWIW. I'm only 39, but I really don't foresee myself having any problems riding fixed. I could be wrong though.