Yep... Did a metric century the first weekend of the month, followed by a century the next weekend on my Surly commuter/utility bike. Just to rub it in a little, I left the panniers on it. I wasn't the fastest rider out there, but I certainly wasn't the last one out on the course, either.
Rode with a small pack for the first 15 miles or so, but as they thinned out on the course and riders split off for the shorter routes, I did the vast majority of the ride by myself, in the wind. On the metric, we had a small group of about 5 riders, but guess who got to "pull" almost all the way around? The guy on the loaded tourer ;-)
Ride _your_ pace, and don't try to get up there and hammer with the bunny rabbits, and you'll be okay. And you'll have a few things to brag about as well...