Plus a man's sleeve length will probably be longer and the waist will not be as small in proportion to the chest/shoulders as a woman's would be.
If you go by bust measurment to size a womans into a mans size, make sure to find out the waist size on the mans to ensure that the bottom will fit over the womans hips. This is not to say that all women have larger hip/butts, but for some of us this is a consideration

!