OP - If you're riding in Zone 2 and were intending to keep your HR or FTP in this zone, it can be very annoying when someone latches onto your wheel. I have a tendency to ramp it up to have a little fun and give them the challenge to see if they can hang on. Then this blows your original plan to stay within the zone and now you're doing their ride instead of the ride you intended. I'm getting better at ignoring this situation and just sitting up and let them pass to get back to riding what was planned. If it happens to be an LT ride, then let the fun begin.
Riding unannounced behind someone is a dumb thing to do, but unfortunately it does happen. I usually either hear them or sense they are back there, so I don't get surprised too often. If someone can approach you from behind, that means they can match your speed, but might be doing a recovery to catch their breath after pushing to catch you. Or maybe they're much stronger and are sizing you up. More often, when you pass someone at a slightly higher pace they'll often try to latch on. I don't mind this situation since you know they're there and it often turns into a fun ride with a shared effort. If I notice they have poor riding skills, I'll either ride them off my wheel or just sit up if I can't.