Alternatively, you could ride a 24" bike. They use the same components as other bikes when/if it comes time to upgrade and you can even get quality tires in 24" sizes.

Specialized makes a cool, rigid MTB in their Hotrock 24.

If you prefer road bikes there's the Fuji Ace 24.

Either can be had for under $400.