I haven't ridden the Serfas, but I have had several Brooks saddles. I suspect you're going to have a hard time adjusting if you're comfortable on the Serfas, they're very different saddles. Personally, I find "comfort" saddles like yours to be too soft for my liking, and they make me go numb.

As is usually mentioned in these saddle threads, anatomy differs from person to person, so it's difficult to know what works for you until you've tried it.

I'd suggest giving the Brooks a shot, and if you can't stand it, put it on craigslist and recover most of your money.