A lot of my PBs on short climbs in town have been done on my fixed gear just because you don't have a choice in the matter: it's either mash it out and get over the hill, or walk the bike. There's no bailout gear to let you take the hill at a more comfortable pace.

I ride my road bike when I get out of town in the mountains. My road bike rides have a lot more climbing in them, so the average speed might be lower than I normally cruise around town on my fixed gear. Are you comparing average speeds over the same course on both bikes? If so, I would say the drop in speed is due to not pushing yourself as hard with lower gears available to you, as others have stated already.