If you're just putting a small amount of weight up there is shouldn't be a big deal. If you start carrying a bunch of weight I'd be concerned. Ditto's to what's been said already.
Don't forget, the more space you have to carry stuff, the more stuff you'll carry! Whether you need it or not...
