You should go as far as you feel comfortable with. It will depend upon how your overall health was before, and how much you have actually recovered. Remember that taking a few weeks off will make you a little weaker especially with a severe cold/flu.
Don't over do it, and make sure you have been very well hydrated. Having a cold/flu but will not help you be hydrated.
__________________
2007 Jamis Ventura Comp
2006 Jamis Explorer 2.0
2000 Specialized Hardrock (bought used)
Swim, Bike, Run and sounds like fun