Is there a significant change in handling if I swap out an 80mm stem for a 120-130mm stem to get a more comfortable crouch in the downhill turns?

Backstory:
My main ride is an all-terrain touring bike in 650b. As with most bike brands, I straddle two size ranges.
And as a result i usually end up just a tad cramped when I'm in the drops-- especially if I need to get low on descents.
It's a gravel touring bike, so I ride accordingly: party pace to all out on the flats, steady up hill, and (very) conservative on the descents.

Thanks for any input, advice, and insights

Changing your stem for one which is 40 or 50mm longer will cause a significant change in handling. What we need to establish is whether it would be a good change or a bad one.
Regardless of your dimensions or position or geometry of the bike, your steering will slow noticeably. Now, if the 80mm is already too short and twitchy, going longer may be just what you need. Are you cramped in all positions with the 80mm stem or are you just not used to bending your elbows? However, consider that a) you will be potentially stretching further over the front, increasing the angle between your torso and arms which will require a little more core energy to support over time and b) although rideable you may not be able to bend your arms enough when just cruising which could tire your neck and shoulders.
I find there’s quite a fine line when selecting the ‘right’ stem length although most people can adapt within a range of about 30mm.
Without seeing you riding the bike and assuming your current fit is in the right ballpark for the sort of riding you do, I would suggest erring on the side of caution and moving in 10mm steps.
Over a long ride I think you’d be surprised what difference it can make.
Obviously it would make sense to buy cheap or used parts until you dial in exactly what you want.

If getting a different length stem makes you more comfortable, then I'd doubt the handling differences will outweigh the benefit.

That said, maybe another new or used bike will be better.

Handling varies a little with stem length, but one very quickly gets used to it. There seems to be no advantage of one stem length over another in terms of performance.

My quick check for stem length is to look in the mirror or take a photo, hands on the hoods, normal riding posture, straight back - your upper arms should be at ~90° to your torso.

Fit rules. You will adopt quite quickly to handling differences if the fit is now better. I rode a 130 stem on a bike that was both large and had a shortish top tube. Noticed I wasn't getting good blood flow in my torso and I would get pain in my abdomen and chest climbing hills. Took a crazy risk and had a framebuilder make me a 180 stem. First ride was a tough, hilly 75 miler. Bike felt very different but no chest pain!

That experience put me on a run of longer stems for all my bikes except the two customs that were built to not need them.