The further back it is, the heavier you'll be sitting in the saddle. Or maybe it's better to think of it as you won't be getting as much up force from the power of your legs to lighten your weight in the saddle. So sure, I'd imagine it's possible.

Though if you aren't being pulled forward when you pedal hard then maybe you are in the correct position for the way everything else is currently set. And might need to check things like saddle tilt. Or even need to again play that fun for all game of saddle swap! <grin>

You haven't also switched brands or model of cycling short or bibs have you? I'd think a thicker chamois might put some pressure on your perineal area compared to what you have your saddle tilt set for with a thinner pad. I prefer the thinner chamois that are usually found in the mid to upper price models of bibs/shorts.