It takes me at least 5 miles to start feeling warmed up. Unfortunately, we hit a hill about 3 miles into many of our rides.
Because I usually ride with a group, I don't go home early on a bad day, hoping there is someone else in the group who will be slower than me that day.