if you have a proper strength routine and as addicted as you say, you will maintain all of your strength with a 2 day a week workout. You probably will lose a few lbs but if your nutrition is on point, you won't lose that in muscle.
I think guys are selling you short saying your still 205 and will always get smoked. Yes, you probably won't beat anyone on the climbs but I know how strong my legs are at 6'3 235. If I shed 25 lbs I'm still 210 but I know guys would be in deep **** against me on flats and downhills. You get yourself on a properly fit bike and get those legs strong and you can surprise some people, your not really heavy at all.
BTW, I understand what you mean about not wanting to get super skinny like you used to be but 1. your 6'5" you border on freak of nature status already simply for being that tall regardless of whether your 180lbs or 250lbs
2. chicks dig tall guys and would rather a guy too tall than too short, you'll be last on the climbs, but first with the chicks