I don't think some drivers know what a hand signal is -- they might just think you are riding in the middle of the road. I have had the same thing happen to me.