I thought you lived somewhere in SoCal? Isn't the terrain there fairly hilly?

Also, are you tracking anything other than speed on these rides? If you use HR, how is your average HR varying between those two types of rides? If I'm "cruising", my average HR for 3-4 hours would probably be ~130, while if I'm going for a sub 1 hr kill myself ride, my average HR would be ~165. The average speed difference between generic examples of those two rides would maybe be about 3-4 mph for me, assuming similar terrain and no other riders or crazy wind.