I only use the drops when I get tired of fighting a headwind, doing the fairly rare speed segment, and to give my hips a slight change in position, which is the biggest reason I find myself in them. I have a bit of arthritis and for whatever reason, it's an almost immediate relief in pain when I go to the drops for a few minutes.

Most riders are in and out, more in than out of the drops. Just spend a little time in them here and there and the muscles will develop soon enough.