There were cyclists ahead of me which I started overtaking. As I passed on he tucked in behind, then another and I had two riders sitting in. After a 2 mi I pulled out and signaled for them to come thru and started to drop back. They slowed down! I couldn't get either one of them to take a pull and had play locomotive for the next 2 1/2 miles.
Sounds to me like they were unable to keep your pace except by drafting you. You were the stronger rider and they were incapable of pulling you at a pace you would like. Let them draft, or don't, as you wish, but I think there's nothing they could do about the fact that you wanted to draft somebody.