You might think that the 650c bikes are the same geometry, but if they're well-designed frames, they are definitely not the same. For example, wheelbases would be tighter with 650c frames so the bike has quick handling and not feel "big". The rear-tyre would come just as close to the seat-tube as on your bike. However if you were to put 650c wheels on your bike, you'd have a 1" or larger gap.
The other change is bottom-bracket height off the ground. The 650c frames has roughly the same BB-height as 700c frames. This is done by making the chainstays close to horizontal with the ground. If you were to put 650c wheels on a 700c frame, your BB would be 1" closer to the ground than before. You'll risk striking the pedals in corners and high-siding.
Finally, the front-fork geometry is different. Due to the shorter radius of the wheel, 650c frames tend to have less fork-rake in order to have the same 50-60mm of trail for stability. Putting a 650c wheel onto a frame made for 700c wheels would end up with less trail and twitchy, unstable handing.