If you are riding fixed, I'd go with the smallest you can find. I had large but I kept hitting my foot with the front tire. This can be a huge problem. I don't really know that the size of your foot has much to do with it.
__________________
Well at least I'm housebroken.