I'm sorry, CdCf, that calculation is idiotic. Or lets just say, incorrect. First of all, 1kg on the bike != 1 kg on you. To take a trivial example, when you start from a red light and stand on the pedals, the heavier you are, the bigger weight is pushing down on the pedals (the more weight you have to get moving as well, of course). Now if that extra weight is on the bike, you will have to get more total mass moving, but not more power to get it moving with. A heavier bike is more difficult to bunnyhop and generally move around as well.
Now, as we all now (khm...) the outer perimeter of the wheels is where weight matters most, as you ave to spin them up every time you accelerate. Lighter wheels do matter.
To OP: I wouldn't do it. You sort of gave up on weight when you bought those silly heavy rims, and there's no going back. Sorry.