You are right, the Tour de France must be tougher than a regular marathon. On the other hand, there are some ultra marathons too.

Actually I have heard that Garry Kasparov, the best chess player in the world, is extremely fit. He must be able to concentrate for several hours without loosing focus. So maybe one should start playing chess if one wants to do something tough.