Hit the yard sales (early) and thrift stores to find a nice old beater. Usually all they need is new tires, a new chain and some lube. Here's a Raleigh Record I found for $10 with a totally rusted chain. I cleaned it up, added a little touch up paint, a new chain and lubed all four main bearing surfaces.

Since this pic was taken I've added new tires and a rear light for night riding. I'll soon be upgrading from this bike to an 83 Trek 400 with aero brakes, also found at a yard sale.