You're going to have to lose some upper body muscle if you want to get down to a sustainable lower weight.
If you don't like the idea of that, then target flatter road races, crits, and events on the track. You should still road hilly RR's but that's because you friggin' love the suffering.