Scouser,
Welcome to C&V. V-Brakes will not work on a Varsity. V's need special threaded posts which are attached vie welding to the fork and rear seat stays. Unless you have a frame builder weld them on (which will ruin the paint in those areas), they will not work.
Varsitys came with caliper brakes similar to the one shown on my '62 Continental.
You could also probably equip it with center pull calipers with little trouble, which might provide better stopping power. If you would like, I could dig through my spares to see if I can come up with a suitable pair I could send you. Do you need levers? PM me if interested. Do you live in a hilly area?
Best of luck. Can you post some pictures of your Varsity?