Yeh sort of, but its got more to do with clearance. If your dirt jumping you want to be able to move around the bike without the top tube getting in your way. As for XC your right, better pedal efficiency. Just look at road bikes, the top tubes are almost parallel to the ground.

Then there is the different head angles etc. For instance a DH bikes usually has slacker head angle which helps controlling the bike at high speeds, where and XC bike will be different again.