Good policy. I'd stick with that.
Before reading Grant Peterson's book (actually more a collection of blogs printed into a thick pamphlet) I used to make each ride's average speed 20mph, and took pride in finishing my centuries in less than 5 hours.
But what's the point of driving to someplace beautiful like Solvang, Sunrise Highway or Death Valley to ride 100 miles if you're going to spend much of the time looking at the little digital display, and making that the primary determinant of your experience?
Even if you buy a power meter and train your guts inside out, there will always be faster riders.