Speed outside is nearly useless as a metric for this reason.
I'd recommend getting a trainer like the Kurt Kinetic Road Machine and attach it's $50 power attachment if you're not up for a power meter. Or, if you already train with heart rate, hit the trainer, warm up to a goal heart rate and ride without watching your speed - just heart rate. This is at least a way to control the environment.
Or... if you want to stay outside, do a hill climb. I have a hill climb that I do where wind won't be so much a factor. It takes me almost 30 minutes to complete the climb, and it's nearly always on the "leeward" side of the mountain, so wind isn't an issue. 32 minutes in October, 29 min 54 sec in November, 1st of January was 28 min 51 sec. Next Tuesday I go up again.... ugh.
__________________
Envision, Energize, Enable