Talking on a cell phone while driving, whether bike or car, is unsafe. People who do it are endangering themselves and others. They should "shut up and drive" as the bumper sticker says.
Anyone who is upset that you're not a slave to your cell phone needs to spend some serious time adjusting their own priorities.