I used to commute nearly every day, a little over 5 miles each way. When I started, my time was over 24 minutes (urban cycling, not a lot of elevation gain).

First week ride: 2/28/2017 Avg. Spd: 12.1 Time: 25:59:00 Distance: 5.2
Third to last ride, before changing locations: 12/21/2018 Avg. Spd: 16.2 Time: 22:04:00 Distance: 5.2
So, over a year and a half or so, my average speed increased by 4 mph. The drawback is that a 20-minute ride is really just getting warmed up.

I now have a 45-minute commute with five times the elevation gain, so I can work on my climbing, if nothing else. Commuting certainly helps fitness - it's the only way I'll be able to squeeze in a hundred miles a month to get ready for the July 4th climb up Mt. Baldy here in SoCal.