Not over-simplifying things. In the case of a rider that is not concerned about speed, one of the most determinant factors here would be the weight.

First, a single speed bicycle has a much straighter chain-line and does not need to disperse its energy throughout a derailleur drivetrain. If you want to think about this concept in terms of gear inches, the single speed is more efficient. There is far less friction on the single speed.

Additionally, there is less mass to be moved in the rear, which makes start/stop easier. We must also factor in wind resistance and rider position. Already we have an easier start/stop over a geared bike.

Finally, it is impossible to measure fatigue since it is perceived mentally. The rider might just have an easier time regulating their speed through the simple concept of pedal more = faster, pedal less when tired.

If the OP is trying to go faster than 17 miles an hour (which I highly doubt given his "condition") over a distance of more than 8-10 miles, then I rest my case.