I started with a Sirrus Sport, and went to a much more aggressive geometry flat bar road bike. My avg has gone up slightly, but nothing like 3.

Realize that an $800 Allez has the same wheels as your Sirrus, incredibly similar frame and overall weight, with the benefits being drop bars and shorter chainstays. It is more aero, which comes into play the more time you spend over 20mph (I have heard 18 too).

Unless you are doing 10 laps around your lake, keep riding your Sirrus. Push 50 mile rides, do intervals, and hill repeats. Your speed will jump faster that way.