An argument in favor of granny gears that aren't super-low
Since I'm in the minority...
At a certain point, I'd say that lowering your gears not only hits diminishing returns, it's actually counter-productive.
Let's say you are on a hill that is 1 mile long. If you're pedaling at 4mph, that hill will take you about 15 minutes. At 3mph, it will take you 20 minutes. I'd say for many people that's a good trade-off, although you're working 25% longer.
What about a longer hill, say 5 miles? At 4mph, it takes you 75 minutes; at 3mph it takes you 100 minutes. Now, you're fighting gravity for an extra 20 minutes. Is it still worth it? (Before you answer, try to recall how you felt when you recently got to the end of a 5-mile climb in your lowest gearing.

)
How about over 10 miles worth of climbing? 4mph is 150 minutes, 3mph is 200 minutes. Even if that's spread out over a 50 mile ride, a slower pace will mean you have to climb for an extra hour.
20 miles? 4mph, that's 300 minutes of climbing; at 3mph, it's 400 minutes. I'd say that adding over an hour and a half to your day's efforts is quite significant.
How about comparing a 4mph pace to 3.5mph -- can't be that much, right? At the 20 mile amount, it's an extra 45 minutes of climbing. So lowering from that 26 to a 24 up front, which is a very small change, is going to mean you work longer on every climb.
Keep in mind I'm not characterizing this in terms of
performance or presuming that saving time is of primary importance. Rather, I'm pointing out that almost any mechanical decision involves a compromise. If you choose to work less on the climbs, then you'll need to work longer -- even when you happen to be cycling at slower speeds.
Thus at some point, it's better to get stronger than to lower your gears.