Shortening stem length will make a small difference in handling, but, not enough to justify tolerating a bad fit. Get your fit right and the bike will be more comfortable to ride and you're highly unlikely to notice any handling changes.
__________________
Birth Certificate, Passport, Marriage License Driver's License and Residency Permit all say I'm a Fred. I guess there's no denying it.