I run a 17 on one side for general road riding, and interchangeable 14-15-16 on the other side using Miche cogs. In the past I just used the smaller cogs for track, but I might start using the 16 or 15 on the road sometimes since I live in a flatter area now. As a rule, I don't flip my wheel while out on a ride. If I were to do a really long climb and descent fixed, it would probably be worth it to change at the top, but that's not really my thing. Depends on your terrain and riding really.
And you can probably do more than a 2t jump. I think someone calculated that a 1t difference equals like 1/8 movement of axle in the dropouts or something. I regularly switched from 17 to 14 with plenty of room to spare. How big of a jump your bike can do depends on how long your dropouts are.