So is do you hear/feel the rumbling only when the bike is under its greatest load? Have you tried to spin the bottom bracket/cranks without the chain on to see if you feel anything odd?

Given that you're experiencing this problem in only some gears, I'd suspect your freehweel before the bottom bracket; if some freewheel cogs are worn, you might experience that roughness.

Neal