Thanks everyone for your replies. I know that ultimately I'll have to ride both to decide, but it's interesting to hear everyone's opinions. Luckily I have a flip-flop hub so I think I'll start with fixed and then have a freewheel on the other side in case I'm not comfortable with fixed or just want to try something different.

90% of my rides are very short distances so I like the idea of getting a little extra workout with the FG.

Question: I'm putting aero levers on. I'll be using the front brake with FG, but not the back. Should I just put the housing of the lever for the back brake on anyways for hand position? For those of you who flip between FG and SS, do you only use a front brake even with the freewheel? Or do you have a front and back brake with both FG and SS?