I think you're doing it wrong.
Please see the attached image for the correct way of riding drop bars on a fixed-gear bike.
Don't worry, it's a rookie mistake but everyone here has done it at some point or another!