You may want to check out crossfitendurance.com for a system of combining CF & endurance training. The site is not terribly clear, but if you poke around you will piece it together. The basic idea is to still do CrossFit 4X a week, but most of your rides will be short, mostly either interval or tempo training. This allows for more recovery, which we do need. Each day the site posts a workout for runners, swimmers, cyclists, and C2 rowers. On "tempo" days, there are different distances depending on the length of the event you may be training for.
The CrossFit Endurance method is controversial, since it flies in the face of the "long, slow, distance" philosophy of a lot of endurance training. And IMO there's a valid argument that CFE alone will not totally prepare an athlete for events longer that 2-3 hours -- though the creators of CFE insist otherwise. But CFE is a great way to get strong on the bike while keeping your all-around CrossFit fitness.
I'm also a CrossFitter and am getting back into cycling. Since I need to harden up my butt for a long event soon, I'm doing long-for-me Saturday rides in addition to shorter CFE rides during the week. For now, Sundays are for rest or very easy recovery ride. When I get closer to my event (a 4-day, 220-mile ride), I'll start doing consecutive long days and ease up on the CrossFit.
Good luck!