Didn't major in physics, but here goes:
The more circuitous route up the hill takes more energy. That's because of the extra wind and rolling resistance from having traveled farther.

The phenomena you're describing are physiological. If you were stronger, or could work your legs efficiently at a lower cadence, the hill would be easier and faster done in a straight line. Stronger legs will take you comfortably down the fall-line too. Time to hit the gym.