Is there a significant change in handling if I swap out an 80mm stem for a 120-130mm stem to get a more comfortable crouch in the downhill turns?

Backstory:
My main ride is an all-terrain touring bike in 650b. As with most bike brands, I straddle two size ranges.
And as a result i usually end up just a tad cramped when I'm in the drops-- especially if I need to get low on descents.
It's a gravel touring bike, so I ride accordingly: party pace to all out on the flats, steady up hill, and (very) conservative on the descents.

Thanks for any input, advice, and insights