I get where you're coming from. I bought my Brompton to replace my old FlyBy. Wanted something that folded easier and was lighter. The Brompton met both of these requirements,but I'm not completely happy with the ride,the shifting is weird and gear spacing odd,and pulling the rear wheel is way more of a PITA then it should be.
Look at the bright side,how many rim brake pads would have needed to swap(and wait till you see how easy replacement is)? Would you also need a new chain/cassette if you were running a der setup? And I think you'll be glad you have the rack if you ever need to haul something heavy. Much better on the bike than on the back. As for carrying a pannier,maybe you'd like
this. Saw one at REI awhile back,seems to be a good balance between on and off bike.