In addition to the aforementioned spare tube and pump and levers, have the know-how to use them. Practice taking a tire and tube off at home, so when you have to do it it isn't a learning experience. I was about ready to help some folks on an organized ride last fall, until SAG showed up, who had everything they needed but no clue how to use it. If they'd have been on their own, it would have done them exactly no good.

Past that? MY must haves are gloves and a basic wired cyclocomputer. YMMV, though, but those are the two things I always have. Helmet also strongly recommended, as I've learned everytime I've crashed without one, the only thoughts going through your head at that moment are "HEY IDIOT, DON'T HIT YOUR HEAD!"