Assuming both bikes allow you to carry whatever you need to finish the ride, I don't think it really matters. You should decide based on how you want to ride.

looking at it another way: Why is the touring bike slower?

That's partly a rhetorical question, of course. I have found I go considerably faster when I ride a fixed gear bike. It's not that the bike is faster, but that it's unforgiving. It won't let me gear down and spin up the hills, so I attack them and get over them faster. It's not more comfortable, and it's not easier. The bike isn't faster, but it definitely forces me to ride faster.