It's possible you're slower because of the smaller tires, but in that case you just push a harder gear and it's all good. How big are these climbs you're going up?
Although, there's no reason you should be going slower down the hill, save maybe the tires breaking in a bit...