I have a bike that has a top tube of 52 cm based on 2 LBS's recommendations. Don't you love it when they can just look at you and say "Hmmm, you need a small frame."? Anyways it turns out that despite my height I really need a 54 cm. Anyways I was able to adjust the bike to finally fit me. It does look a bit weird that the seat is set backwards so much but my knee pain went away and I'm able to ride the bike instead of selling it or getting a new one.
Bottom line,IMO, as long as you can adjust the bike to fit you its fine. Even if it looks too small for you.
EDIT: I also read somewhere that TDF riders purposely ride smaller bikes for reasons that escape me right now.