I usually reply "well then, stop making it so dangerous."
Seriously, when someone says this I assume that it means that they realize that THEY don't drive safely enough and that they could easily hit a cyclist.
__________________
Work: the 8 hours that separates bike rides.