Realistically, you if you are serious about racing, you will be on the training inside of a week (no hands). You could be racing as early as four weeks depending on severity and separation of the fracture. Take care of it in the early stages (read: NOW) and you wont suffer too much of a setback.
If there was quite a bit of separation it may be a while longer. Realize that in three to four weeks while it will probably be functionally stable, it will not be healed and could easily be rebroken with another fall, so be careful.