People who become famous always are hated and not liked by some, it doesn't mean said person is bad. From what I've seen of lance, I'd say he seems like a well rounded nice guy. The kind of media attention any celebrity receives is just horrible, it exaggerates any little mistake they make and demonize them becuase of it.
It's amazing how you guys met, rode and talked with him. My cycling friends would kill for a chance to do what you did.