My friends are too fast and my wife's too slow, so ....
I ride with my wife once or twice a week, mainly to encourage her to do it--otherwise she just goes to the gym, and that's so boring she doesn't stick with it. But as DnvrFox said, there's a big gap in our speeds and interest, so I try to schedule that for easy days or days off.
I've ridden with the same group of friends for more than 10 years, but they've all just passed 50 and apparently they feel the cold breath of death on their necks--they've gotten VERY serious, and I can't hang with them for more than a few miles. On organized rides (we do several a year) I sometimes start early with another slow guy and meet them at the lunch stop, or get a ride out to a midpoint and finish with them.
I'm going to go a little harder this year, with some goals in mind for late summer, but I still won't be able to match them. They're mostly self-employed or retired, and can take a few hours off to ride during the day when they want. I have to report to a boss, who's good about an extra 15 minutes but not about an extra half-day.
Most of my daily riding, 125 miles or so a week, comes in my commute ,which I do solo. The longer weekend rides are about half alone, half with others.