I carry both. I prefer the convenience of CO2 but also the potential unlimited air from the pump. As for the waste of the CO2 cartridges mentioned, I do this so rarely it's a non-issue. It's been over a year since I needed to use CO2 or the pump. Knock on wood.
And in fact, I've never used the pump for myself. Just others I run across who have forgotten their pumps or had some issue with them.
I could certainly make an argument to not bother with the pump. But I treat it like a talisman. If I'm prepared, it won't happen, if I'm not prepared, I'm doomed.
As an example, I keep a foot of Gorilla tape wrapped around my pump. Just in case I need to tape something down. Sure enough, for some reason on one ride I forgot that pump. Didn't need the pump, but broke a spoke that was flapping about. I thought, "hey I'm prepared, I have that tape around the pump ... that I left at home. Dang it." Made use of a bit of tape in the small first aid kit I carry to deal with that. Yeah, I'm a belt and suspenders type.