It's good that you can get by with backpack. Expands your bike choices, and, as you said, centers the load a little better. But I wouldn't call a rack "the wrong tool for the job" unless you ended up with a particularly poor rack, which, given your story, sounds entirely possible. A rack's whole function is to carry stuff on your bike, and a huge percentage of tourers, commuters, and utility riders use them to good effect.

In 30+ years of riding, my bike has almost always had a rack, and I've never experienced the failures discussed here. I do think I might have lost a bolt once and had to zip tie the rack leg in place until I got a replacement. Not a big deal. Of course the OP is talking about a rack that connects to your skewer, so the only bolts to loose are the ones at the top. You don't want to loose those, either, but they're really just keeping the rack level rather than carrying the load, so coming up with a temporary fix shouldn't be too bad in the unlikely case of a bolt issue.

Your wheel troubles sound like a poorly built wheel, flawed wheel components, or a wheel that was already fatigued. If an additional 40 pounds (and not really additional, just moved from your back to directly over the wheel) causes your wheel to self-destruct, then that wheel was already a problem waiting to happen or it was simply never made to carry the weight you have, and you've just been lucky up until now. I've had spokes break, and they can sometimes have a domino effect: One broken spoke increases stress on the other spokes, and now they are more prone to failure. And certain spokes are more prone to failure than others. I had a wheel that, after the 3rd spoke broke on the same side of the wheel, I just bought a set of well recommended spokes and replaced broken ones as they failed. Eventually all the spokes on the problem side were new, and the wheel was solid.

I've used a backpack many times for different reasons. It's manageable, and for the people who prefer it, that's great, but it's not my preference. Switching components between bikes meant that yesterday my commuter had no rack, and I had to use a backpack. It did the job, but when I got home, I made sure to put the rack on. Also when I went to leave work, my shirt was still damp, which is unusual. It's not that a rack means that you don't sweat, but a backpack means you sweat and it has nowhere to go. Usually I'm dry within 15 minutes of getting to work, but the wet patch under my backpack never had enough time to air out. It's not the end of the world. It just feels better to me the other way.

To the OP, I would say to pay attention to the published weight limits on your rack. Make sure your wheels can take the weight, as you would for any touring load, and, if you're really paranoid, bring a spare skewer. Know that your rack, like any other part of your bike, can fail, and, given long enough, will fail, but you don't often hear horror stories like bikenh's. A decent rack and a little attention and you will likely be fine.