a right hook deserves a smack on the sheet metal. It does make a resounding boom inside the car. In general, I prefer to use my open palm, no chance of damage to car or hand. I have always figured out someone was going to right hook me in time to make the turn with them. I guess some day my luck will run out.
I had some guy race me to a stoplight down a hill in a 15 mph zone. I just rode further and further out into the lane. He must have been going about 40 by the time he realized he wasn't going to pass, I'm sure we were riding at least 25. There is almost zero chance of hitting this particular light on the green cycle. I don't let cars pass me at that light or on the next stretch of road because there is no chance of catching the next light on the green cycle either, and the road is covered in potholes. I ride it at the same speed they would drive anyway.
In general, I let cars pass me on the way to a stop sign or red light. It is better to have them in front of me in almost every situation. I try not to think about the fact that they are saving themselves zero time.