Funny dilemna. When I was a big time bike racer I weighed 156, had 9" arms and about a 27-28" waist and was scary fast.
Now my arms are double the size they used to be and waist is about a 34 and I am 72 pounds heavier and if I were on the bike I would just be scary...
There is a tradeoff for sure. When I raced at 200 I was very quick but focussed only on crits. If I were to ever come back I would probably race at 210 as I would not be prepared to sacrifice all the muscle I have put on owing to the fact that this is just for fun ('fun' as defined by me might be a little different than most) and would not have any interest in roadracing against 145 pound cat 1s with too much time on their hands.
I also enjoy being able to do other things in life (I am a very good golfer and hockey player) and having the size/muscle definatly helps...