At what point does this start to matter? At one end of the spectrum you have folks who are just out for a leisurely ride, and at the other end are the hard-core racers. But in between are the fitness riders... and even in that group there is probably a pretty wide variety. So my question is whether it matters right away while you're new to the sport and still getting in shape. Or do you reach a point (after you've been biking a while) where things like shoes, more than conditioning, are holding you back?