I have been in competitive sports for a long time and in my experience the moments that stick with you are not always those that result in a W.
I know this will sound lame, but one of my best sporting memories is coming in from 3rd base in a Little League game down 17 to 0 with one out in the bottom of the first with one out to pitch. We ended up losing 21 to 17 and I pitched the rest of the game. Not only did I get a game ball in a losing effort (for the first time that I know of for that coach), I was also put into the starting rotation for the rest of the season and never disappointed.
Truth is that I had a huge amount of upside in baseball as a youth compared to what I could do in bike racing. There really wasn't much money in either spot way back when.