I don't like strangers riding on my wheel, and I don't jump into groups. I appreciate them asking, and sometimes awkwardly oblige, but I don't like it.
Not asking is plain rude, and can be dangerous.

-You are more likely to get in an accident riding with those of unknown and different ability. One person's casual talking speed is another on-the-rivet speed.
-There is an implied responsibility of the rider/s in front to point things out/ride a particular line
-If you are in dialog with someone, you may/may not want someone else listening in.
-If you are truly training, in TT mode, there are wind and other factors that play by having someone on your wheel.