The first rule of self-defense is to avoid a fight if at all possible. You did well. If he got out of his vehicle and invaded your space, that would be a different story.