This is my second season spending significant time on the trainer, a Cycleops Fluid 2. For the second time, I noticed about a 3mph drop from the road speed that I was averaging to the trainer for rides of comparable length. That 3mph came back as soon as I hit the road in earnest in March.
I don't use or own a power meter, but just for grins, I ran the power curve calculation using the Cyclopsops formula I've seen published in another thread. On a recent 75 minute trainer ride, the outcome for my 19.1 mph average speed on the trainer was 246 watts. Plugging that wattage into bikecalculator.com for a 160 lb rider on a 16 lb bike, flat, no wind, hands on the hoods, gave an average speed a shade over 22, which is pretty much where I am on solo rides working moderately hard in those conditions.
All in all, I agree with the OP's observation that a difference exists. Riding on the road is the apple. Sitting on the trainer is the rotten orange. Compare only things of like kind.