Track bikes look snubbed because they have steeper angles, and usually run a little smaller (if you ride a 58 road frame you might take a 56 track). That said, I'm almost exactly the same height and inseam as you, and I'm riding a 57 right now with horns and the seat pretty far back. I think a 54 would crunch you up something awful. Especially since you were sized for a 57 at the LBS.