I ride in the drops a fair amount, most often on long flats. For fast downhills, I tend to be in the bends, covering the brake levers (just in case I get scared...), if I want to be more aero there, I just bend my elbows.
Riding in the drops can be comfortable when you get used to it. These days, my stem is 2 inches lower than the saddle. I think that some people feel an obligation to be hammering when on the drops, which inhibits used of them. Sometimes, when I have been off the bike a lot, I pointedly force myself to ride the drops a bit to get used to it again.
I've also largely given up riding out of the saddle in deference to my aging knees. Maybe it looks ridiculous spinning up a hill in a super low gear, but if the gear is there, I'll shift down to it and stay in the saddle.