I agree, with the right seat adjustment and pedal stroke it only matters if the bike is extremely small or large. Don't get hung up on it, if it feels ok then go with it.

All I can say is, bigger is better.

I can feel more comfortable on a bike that is a little to big, as opposed to one that is too small.

Doesn't the basic geometry change from one type of riding to another? For example, isn't a touring bike generally longer than a typical road bike? And aren't downhill mountain bikes generally smaller for easier standover height and maneuverability?

It does make a difference to me.