To sum up the arguments:

Comfort - There is a comfortable fit for traditional road biking and a comfortable fit for aero bar use. They are not the same. A properly fit bike with aero bars is going to be very comfortable. The only ones who will argue that a properly fit aero position isn't comfortable hasn't had a properly fit aero position.

Speed - You will go faster, use less energy, be more efficient with the aero bars.

Groups - Do not use them while riding with groups, you posted about riding alone, hmmm, that means your not with a group, sounds like your ok here.

Only for TTs and Tris, becuase you would only want to be faster, more comfortable when riding in a race?

Only Use Limited periods of time/Short bursts - See RAAM as mentioned before

Control - yes, when you are cornering, high speed descending or coming to a stop, get out of your aero bars, takes about 5 minutes of riding to figure this one out.

Yes, the only people who ride in aero bars are those who saw Lance ride in a TT or those that like to ride alone, fast, and comfortable.