There is something amiss here. Was the car in front of you, beside you, or behind you when you made your signal. If he was ahead of you then you should have made the necessary precautions. If he was beside you then you may have surprised him and he did not see you in which case you should still make the necessary adjustment. If he was behind you then you are dealing with a Cager/JAM who can't wait 5 seconds to make his turn.