I agree with everyone else on this. No he would not have the kind of fame he has now if he hadn't won in the tour. I dated a girl for a few months and she came over while I was watching one of the spring classics and a few riders that she heard me talk about before were mentioned, she asked what I was watching and I explained, she then explained that she thought that the people who ride in the TdF only ride in that race and spend the rest of the year training for it. I've met other people that are the same way; they think that there is only one real pro race each year. So without winning that you couldn't (in the US) possibly become a household name outside of cycling fans to the point of Lance or Lemond. Same goes for soccer in the US, most people essentially think that there is the World Cup and the rest of the year they just practice for that one tournement.
I think another part of it too is that it seems like Americans are the ones that the race itself dislikes to a point. I think that is more of public perception and not a fact though. But it's similar to Bobby Fischer, he could have won every chess match he entered and not been a household name, it was beating the Russians at their own game so to speak that got him his fame (and subsequently his insane antics and absurd statements that kept him famous (... Greg Lemond?))
Those are just my opinions on it,
Kyle