You shouldn't expect much shrinkage from a regular cycling jersey. Personally, I'm not into skin-tight jerseys, and you can't see my ribs through them. All my jerseys have a zipper at the throat, so I don't have to worry about choking myself.

Go ahead and keep the jersey -- especially if you really like it. Now, if by "it's a bit flappy" you mean it resembles a giant potato sack, maybe that would be a little silly.