My guess, is that if it was very loose fitting, it may feel a bit distracting over a long period. I have always preferred jerseys/tops that hug the skin, but not "tight", just enough so it is not flapping in the wind. Generally, a good wicking fabric tank jersey, or shortsleeve jersey works best. For the ladies, I would always recommend a good wicking sports-bra underneath, even if the jersey is a good fit. Unless of course, it is a jersey or haulter that has a bra built in.

BTW, the response above is quite mature and helpful.