I'm about your height and I find vintage steel bikes are great. Sturdy but light enough to give a fun ride. No need to go absurdly huge, like a 67cm Fuji, but they are out there if you want them.... My good friend has a larger Peugeot that has become his daily rider and he's at least 6'3".

A Varsity isn't a great model for comparison. If you get something with Reynolds or other high quality tubing you won't find them sluggish or heavy.