I don't ever ride without bottle cages, but I prefer riding a bike unadorned with accessories.
I'll run a blinky and headlight clipped on my seatpost and bars occasionally, but the world's lightest pump lives in my pocket, rubber-banded to my spare tube as ballast to keep it from flying out.
A pocket tool or two also predictably fits in my jersey pockets as intended, these can be quite small and light!
Here's what fits in my three pockets, with room to spare. Even my windbreaker fits in. The keychain is as light as it looks, and that little cable retainer tab thing on it doubles as a screwdriver for the limit screws on most derailers.
I think the bikes look better pro-style, with nothing added that can loosen, rattle, fall out or fall off!