i recall a story about the new owner of The Hour record. He's this 2 meter Czech guy. Many say he won it because he's so tall. In sprints you can get as low as you want, so wind isn't an outstanding factor. Also, in time trials, the key is to maintain a good average speed, and being fairly heavy is good for that.
But in any situation where you're out of the drops, being tall probably isn't so great.