Riding shorts with a "chamois" pad. These prevent your undies from getting sweaty and bunching up, which WILL happen even in cold weather, and cause serious chafing problems.
Almost everything else can be gotten from a thrift store (Goodwill, etc). If you have to have new, Target has pretty good wind/water-resistant stuff for cycling.