I use a stainless steel Klean Kanteen most of the time, but on my fixed gear I don't have any bottle cages so I wear a hip pack with a water bottle holder and for that I prefer the fit of a plastic water bottle. As someone who uses both stainless steel and plastic bottles, I can say for certain water tastes much better coming out of the steel container.

My water might get a little warmer in the summer in the steel bottle than in plastic, but I don't mind drinking warm water...but warm water in the plastic bottle has a strange taste. Also, I like the fact that if I'm camping, I can heat my steel bottles for hot water if needed.