I completely relate to your disgust with the abysmal conditions of San Francisco streets. You have to wonder with all the tourist dollars pouring into the city, why a significant portion hasn't gone towards fixing the streets. Welcome to San Francisco.

Riding a 20"-wheeled bike with an aluminum frame as stiff as the Swift's can really make your body fell beaten-up after an hour of riding through the city. I find that riding SF on a folder is like navigating a slalom course, with constant weaving and jumping just to get around the nasty potholes (some sinkholes) and generally obliterated sections of pavement. The sections of bad road that cannot be avoided requires one to constantly lift up and off the saddle, using one's legs as suspension (this is easier done on a fixed-gear because you have the force of back pedal pressure). It really sucks to have to ride like this on a 700C bike, so you can just imagine how much more worse it is on a 406 bike. As uncomfortable it can be, I personally find the fun factor still out-weighs the inconvenience, but in comparison with your Raleigh townie and your 'bent, I can totally see where you're coming from.

Anyway, after having been in SF for some time now, I guess I needn't tell you that you can easily fetch double the price you paid for your Raleigh, as demand for vintage fixie conversions is big in this area.