I would suggest swapping out the rear derailleur since that is where you would see the biggest improvement. The unit that was used on the Varsity model worked well enough but was REALLY heavy and in my experience a bit sluggish.

Have you considered bar end shifters instead of down tube shifters?