Yeah, I don't like backpacks for the same reason. As long as you have two sets of mounting holes/eyelets on the rear dropout you should be able to use both mudguards - fenders in USA - and the rack.

If not: on one bike I had only one set of holes so I used small p-clips to attach the mudguards' struts to the rack itself. It worked, and the panniers covered the ugly hack anyway!