The shorts are a must. You may get away with gym shorts, but if they start rubbing wrong remember they will keep rubbing until you get back home. Once you find shorts you like get a second (and perhaps a third or more) pair. You always want a clean pair.
Real cycling tops are a big advantage, tehy are cut for the position on the bike so you are not showing a plumbers smile white riding. The pockets in back are really nice, especially once you start doint longer rides and want to keep some food somewhere. My one rule is BRIGHT. There are some nice cheap single color flurecents out there, but if a car ever hits my I want the driver to claim he was blinded, not that he did not see me. Keep your eyes open. Last years team jerseys sometimes go sheap and often have good quality. (A cheap zipper can be a real bummer, full length does little good if you can not work it with one hand).
Shorts are the priority, but if you live where it is hot remember that a soaking wet tee shirt is no fun. Again if you live where it is hot think of investing a bit more in a top that has at least a half length zipper.
Helmet has been mentioned. Think about gloves. Proper cycling gloves have a little padding which helps damp out road vibration. It makes a difference after 20 miles. Pump, tubes and knowing how to change them is a must. I'd personally advise a multitool of some kind. But a cell phone works too as long as you have someone you can depend on.