Single lane road with no passing allowed? The problem with that is that people are going to get pissed off because they will believe you are slowing them down and there is no way around you except if they illegally move into the oncoming traffic for the pass, or try to squeeze past you and risk sending you flying into the shoulder similar to what happened in the TDF this year.

It is still doable though. Just try to develop sort of a thick skin as you will probably be honked at every single day.