Don't give up the gym work. It's something you enjoy and it has health benefits. In addition mixing the two different exercises on alternate days will allow for recovery and help you to avoid mental staleness. For instance you might visit the gym on Monday, Wednesday and Friday while riding your bike on Tuesday, Thursday, Saturday and/or Sunday. Cycling generally requires a lot of time so pushing as much of it as possible to the weekend better fits most people's personal schedules. If your goal is to complete a brevet then you should try to complete one long ride each week, probably on the weekend if that's when you have the most free time. For the weekly long ride start with a distance you can complete and then gradually increase that distance from week to week until you can ride at least 50-60 miles.
Some people take one rest day each week but since you are "cross training" you may not need one. Pay attention to be sure your body is recovering properly. If you do decide to take a rest day I think a short, easy ride is better than doing nothing, unless it's raining.