Bikes are accelerating and decelerating constantly - especially in group rides. But even in a straight TT there are areas weight matters.
In physics we are all taught about how it takes no force for something of any mass to the same velocity (except for friction - air yada yada).
But bikes don't go in a straight line. They travel (trace the path on the ground) in sin waves with the front wheel having slightly larger amplitude than the rear and smother riders - move less.
A wind gust will also move a bike, but the massive one less, and the rider recovers to get back on course using about the same energy heavy or light bike. But the road variations generally don't care about the mass of the bike so much and jolted the bike 2in to the left (complaint tires minimized the movement through shock absorption and reduce the amount the bike it taken off course - another topic) . The rider now has to correct for that. The lighter wheels especially, and lighter bike can do that more easily than a heavier one.