Are you comparing end-of-season times with start-of-season times?
The two most important contributors to speed are going to be the engine and aerodynamics. The big downside of the upright posture of these kinds of bikes is that it doesn't put the rider in an aerodynamic position.
If the rims aren't too wide, you might try narrower/higher-pressure tires (which won't be as comfortable).
Here's a way of estimating how changes will effect speed.
http://www.noping.net/english/
Using the default rider height/weight and a power-output of 160 watts.
Narrow tires: 16.1 mph with an upright posture compared to 18.1 mph
Touring tires: 15.1 mph with an upright posture compared to 16.6 mph
It's unlikely that your short 10-12 mile rides are going to help you get much more speed.