For bushing type, I just pull them apart, clean and oil upon reassembly. If I’m feeling lazy I’ll leave installed and flood the pulleys with wd40. That will almost always penetrate. Some have bearings, I grease those.

Replacements, both of what you contemplate should be fine, as long as you get ones of correct spec. Some may have a different tooth count, pin diameter, or thickness.