Fit rules. You will adopt quite quickly to handling differences if the fit is now better. I rode a 130 stem on a bike that was both large and had a shortish top tube. Noticed I wasn't getting good blood flow in my torso and I would get pain in my abdomen and chest climbing hills. Took a crazy risk and had a framebuilder make me a 180 stem. First ride was a tough, hilly 75 miler. Bike felt very different but no chest pain!

That experience put me on a run of longer stems for all my bikes except the two customs that were built to not need them.