I built some drum brake wheels a while ago as part of an attempt at a low maintenance winter bike. No surprises here: they are heavy, noticeably heavy, they can be overwhelmed by steep descents too. Really they are for commuter bikes.
V Brakes are light, cheap and simple. You could however change to metal cartridge carrier pads like these:
http://www.ukbikestore.co.uk/product...rake-pads.html
My LBS tells me that the metal backing results in the whole face of the pad better pressing against the rim. The best part about them is that once you have gone through the frig of adjusting them - all you have to do is keep feeding them new cartridges. You can get the cartridges quite cheap sometimes - or, I think you can get koolstop cartridges for them too.