There were a few that were downhill skiers, too. Skiing like they do really beats you up but gets you in great shape. But I can't think of any that were XC skiers, which seems a little odd.

And if the OP is concerned with road cyclists, remember many came from the MTB world (Sagan, LeMond, and Armstrong come to mind immediately).