Even average speeds can be very misleading, as on any given ride you may be going into or with, a strong wind. On an out and back ride you'll spend more time on the into wind leg, so it will affect the average more. I guess you have to average the averages to see how things are going!

Just go out and enjoy your rides!