If your goal is to make a commute or a particular route easier it helps to push harder. As your fitness level increases your leasure rides will become easier. However, then you keep pushing and pushing and pushing. Finally you end up riding in the mountains to get an extra workout. Oh the pain!
If slow riding is your thing, more power to you.
If a constant hammerfest is your thing, more power to you.
Just don't be telling me what I can and can't do or how pointless it is.
<--I said that to no one in particular