I'd say try it out. If you hate it you could always run a brake and use a freewheel hub. If you hate that get a internal gear hub. If you hate THAT, sell the bike and get a derailler bike.
If you keep the gear down (I agree you should look into a flip flop hub with maybe a 60-65 inch gear and a 45-50 for little hills and headwinds) you should be fine. I would love to still be using a ss or fixed but I mostly ride to commute and I have to go over a bridge twice a day - it got to be too much for my knees. Now it's all deraillers all the time for me.....if I ever move someplace totally flat I'll def look into a ss or fixed again. (Man, I wish I still had my old track bike!
)