I've tried to resolve similar problems but have not always been successful.

Agree that the wheels and tires could easily be the main culprit, but have also found that gearing is a critical factor, at least in my case. For example, when I swapped out a 12/25 cogset for a 12/27 in anticipation of a hilly century ride, I got slower on my normal training loop. Went back to the 12/25, took off the 13 and put a spacer and a 28 on the back side and went back to being faster again. Those were the only changes (other than the inevitable wind/temp/fitness etc. variables) that I made to the bike.

Have also had some circumstances where changing over an entire component group to a different frame resulted in slower (or faster) speeds and averages. Sometimes I couldn't explain why.