I agree. Anyone grabbing the wheel of any other should announce that. Even if they are there just briefly to wait for safe passing conditions.

Only when riding with others in close groups or formal pacelines will I consider it not necessary. But only for the members of the group or paceline.