Until my left knee began acting up, I did close to half my yearly mileage on a fixed gear bike with none of the accessories you list, unless a water bottle counts as a hydration thingie. (You don't include gears in the list, but they represent a far greater departure from "basic bike" status than aero bars.)

I wouldn't have done nearly as much riding on the fixed-gear bike without the aero bar (plus bullhorns). You might consider trying that setup sometime. It's a lot of fun, once you get over worrying about what other people will think of you.

I rarely had occasion to use the bar tops before I installed aero bars. Maybe that's because I've always been skinny and fit enough to feel no need for that position except for tootling along at low speed. For such riding, it's just as easy to just hold the forearm pads, which put me in a position that's even more upright than that of grabbing the bars next to the stem.

Aero bar weight and maneuverability: anyone who can walk a bike while steering it from the saddle can likely do that with or without aero bars.

The only circumstance I've encountered where aero bars are limiting is riding rollers, where the extra weight on the bars makes it impossible to ride no hands. For me, anyway. On the other hand, it's been 30 years or more since I last rode rollers.

I bought a set of Cinelli Spinaci bars when they came on the market. They came with a pair of minimal/not particularly comfortable forearm pads that attached to the main bar with Velcro straps. Kind of cool, but half the magic of aero bars is having substantial, comfortable forearm pads.

I don't know why people assume that having aero bars requires you to use them as much as possible. Where I live is Baltimore City, so plenty of cars to contend with. Staying off the aero bars in city traffic is easy.