For me it comes down to a combination of comfort and convenience.

My preference is without the water pack, trail or road.
It could be freezing outside, my back would still be sweaty after a good ride if I'm wearing a pack. Plus, it's just more comfortable without something weighing my back down.

If you're a roadie, chances are good you'll eventually come across someplace you can refill your bottle(s)...park, gas station, grocery store. You don't have this option if you're off in the woods. And velocipedio's pointed this out already, but you generally can afford to take a hand off to take a drink. If you're mt. biking, you're too worried about the fallen tree that's blocking your path, much less thinking about keeping hydrated.