Everything is a guess and is going to be off. With a PM, you are able to tell the amount of energy put into the bike, at that particualr contact point and considering the error of the measurments and efficiency of the machine. Unless you have it tested, you don't have much of an idea how efficient your body is producing said power. A safe guess is 25%, but some are more (~30%) and some are less (~20% or lower).

A HRM guesses caloric expenditure based on percieved intensity. For me, I've found that my HRM is fairly accurate, though it's more of a gee-wiz number as I don't do much to count calories.

Also, you don't have an idea of how many calories you body is absorbing from the food intake.

As already mentioned, if your weight goes up, eat less or ride more.