The 50/11 gives you a pretty tall high end. I don't think that that would be a severe limitation. I raced (many years ago) as a masters racer with a 53/13 as my tallest gear, which isn't as high. The 34/28 ought to allow you go ascend Mt. Hood (just kidding, but this is a low gear). I don't see why this bike should prevent you from going fast if you want to and have the leg strength to do so. However, bear in mind that every component on a bicycle is replaceable/upgradeable, including the frame! I can safely say that at some point I've upgraded every single part of a bicycle, usually one piece at a time (although I upgraded all of the spokes on a wheel at once and ball bearings a set at a time!).

That said, if the bicycle is making you unhappy, factor in the "dead weight" of worrying about that all the time when you make your decision.

-soma5