Accurate calorie consumption measurement can only be done in lab environment.
There are online calculators, sports watches etc. that will give you estimates, but they all have to make a bunch of assumptions and are wildly inaccurate, as you have noticed. If you want a slightly improved educated guess, get a heart rate monitor. A HRM gets a realistic idea of your exercise level (through measured heart rate data and patterns), removing part of the guesswork. Still, they are only less inaccurate.
As stated already, muscle weighs more than fat. That may explain your weight increase, at least when you first start a new exercise regime.
--J