It's just the reality of using a vehicle on a public road - we put our lives in the hands of strangers every time we go out. Doesn't matter if you're on a bike or in a car. Ever stopped at a stop light? The guy behind will stop behind you, every time. But not every time.

As others have suggested, get a bright light, take the lane, ride like you own the road just like every other road user. In time the fear will lessen, simply because human beings are very good at adjusting to things over time.