There should be a huge difference. Use this power calculator for a fixed speed, switching from hands on the tops to hands on the bars and see how much more power is required to maintain a given speed. For example, hands on the tops on a flat course for me, 20mph takes 267 Watts according to the site. Hands on the drops, 196 Watts.

So do you have clipless pedals? They give a smoother pedal stroke, for one, thus greater efficiency.