You buy a bike when they think they are too old for a trike. Take the wheels off immediately. Lower the saddle so they can put their feet on the ground - wont take long to learn how to balance. It is easier to learn how to balance on two wheels than it is to balance on two feet.
My children learnt at 4 and 3. It was a real cheap Kmart bike and I couldnt remove the pedals.