Funny enough I do the same with the right hand turn signal, it's a habit I developed from having too many look at me like I'm nuts when I actually signal a turn. If I combine it with pointing in the direction I intend to go, along with a nod of the head in that direction they seem to get it.

except for the guy on his bike who seemed to think I was waving at him ...

one of the reasons we signal with the left hand is so that people behind you will see the signal being given. If I signal a right hand turn, using my right hand, which is on the inside of the corner I'm going into no one behind me will see the signal, it is doubtful anyone in front of me will, so the effort is rendered useless.

in that case I may as well not signal my turns and I don't intend to start doing that.