I feel really slow during the week, when I'm commuting. My bike is heavy, and (especially in the winter) only hard-core riders are out. They *all* pass me.
I feel faster in the summer -- especially on nice weekends. All kinds of newbs out on the road, people just tooling along, or people getting out on bikes for the first time in years. Then I feel not-so-slow. I still get passed, a lot, but there is usually someone slower around on a nice day. All it takes is 1.