Yeah, really wonderful people sometimes aren't compatible with some things we like to do. There's nothing wrong; that's just how things are. It can be very disappointing, though.
Some of the folks I met riding motorcycles came to be better friends the less time we spent actually riding together. And most of the folks that I found easy to ride with never became friends at all.