I use schwalbe marathon pluses in a 25 on my fixie for running around town, just because they are indestructible. And I have them in a 26x1.75 ins on my tourer for the same reason. The fatter the tyre, the softer the ride, most of the time. The penalty isn't really the rolling resistance, it's the weight. Personally I think 25mm is easily enough for ordinary road riding.
But I wouldn't be using marathon pluses if I were interested in performance, or in riding off-road. They are heavy, they aren't great at hard cornering in the wet, and they don't handle loose surfaces very well. Cyclocross tyres tend to be semi-slick with knobbly shoulders, and they give a decent all-round performance. But the general rule is, ride slicks on the road, they give better grip. Ride something wider and knobbly off road for the same reason.