It's good that you and your bike were undamaged, but the motorist was correct and you were in the wrong here. On those types of roads you should be riding in the roadway if you are going to travel at speed. If you are going to ride on the sidewalk you need to move slowly (at a pedestrian pace) and be prepared to yield your right of way for ANYONE.
Only half right, the motorist was required (by most state laws if not all) to stop short of the sidewalk and only proceed after yielding.
__________________
Land of the Free, Because of the Brave.