Wind. My daily commuter is a pretty heavy bike (40 pounds). I have a lighter bike I use for fun weekend rides. If I had to choose between the heavy bike on a still day or the lighter bike on a windy day, I'd choose the heavy bike on a still day every time. I find that once I have the bike up to speed, I don't notice the weight too much. It does take more effort to get going from a dead stop though.

Edit: Now that I think of it, I would also choose the heavy bike if the day is windy. It has a very solid, stable feel about it.