Saddle bag, to put at least one spare tube, a tube patch kit, and a set of tire leavers. (Nothing sucks more than getting a flat halfway between work and home, and NOT having the tools to fix it.)
Flashing red light for the rear, and a headlight that can also flash (I got a cheap set for ~ $12 I think) Mostly worthless for riding at night, but attention getting during my Dawn Commute.
I enjoy having a cycle computer (again with the, I got an inexpensive model, maybe $19) to tell me how far I've gone, how quickly, and at what average speed. I've been keeping a spreadsheet to help me track my personal progress.
Most accessories are things you can add over time obviously. A Helmet, Bottle Cages, and Water Bottles I think should be at the very top of you "Must Have" list.