I love panniers. I'm never going back to anything else. 2lbs per panniers doesn't seem a lot to me. What's 4lbs? Your Fall/Winter clothing may weigh about as much or more How heavy is your bike lock? Your phone, wallet, watch and keys could be close to 2lbs.

Don't be obsessed with weight. It really doesn't make that much difference.