One Midweek group ride I drive to because I can barely get there in time for the roll-out after work, but my weekly Saturday am ride rolls out only 5 miles from my house so I always ride to that one.
I would say most drive if they want to socialize after or live a long way from the start.
Some riders don't have the luxury of decent roads from home to the start too, which may be a factor in them not riding too.