I believe there is more drag going on around the handlebars than we usually credit it, and I suspect that the stem mount, in spite of the optics adds little if any additional drag due to the turbulence. With a particularly aero handlebar and flipped stem it may be a different story.

I've actually been mulling over a side-mount on the stem, to get it behind the bars, but haven't yet because of this uncertainty. This is a trivial enough concern to warrant some testing IMO.