I've got 2 miles of gravel road to ride. Untill last year I felt I had to use a mountain bike with fat tires to get into town.
Then I tried my road bike with 1 1/8 inch vittoria Zaffiros. No problems and a screaming fast ride compared to my mt bike/commuter.
I'm a big guy, too -- 235 lbs.