Keep in mind too that having a more aggressive riding position has a direct correlation with the speed you ride at. At over 20+ mph I notice a huge difference riding on the drops as oppose to riding on the hoods in terms of aerodynamics as well as the lower position giving me a bit more power on my pedal stroke.
If you're toting around at 14-15mph or slower, you won't benefit much from riding with a lower stem. If anything that's where the pain and discomfort comes from. IMO nobody can truly ride "comfortably" on a slammed stem but it's definitely more comfortable if you're riding hard/fast and definitely less comfortable riding at slower speeds.
Doing it just because it looks good or people on bike forums tells you to is no bueno.