+1 on what echotraveler said
+2 on eye protection. You can probably live without gloves unless you intend to ride really long distances or in crappy weather.

If rides of 2 hours or longer are in the plans, add a second water cage and a second bottle.

Depending on your skin type, you may or may not need sunscreen.

And yes, at least one pump is needed. A hand pump would save you money because you can hang it on the bike and you won't have to buy a CO2 inflator and cartridges, and pumping road tires with a hand pump is a good way to build up upper body strength. A floor pump is easier to use, you can get a floor pump with a pressure gauge for about $20 at Walmart.

A basic first aid kit (neosporin, antiseptic wipes, band-aids).