The OP did not say if he\she was holding up traffic or not.
Unless I am missing something....
As for me, my issue was on a street where I was doing this with plenty of room and other lanes for the motorists to pass me in. I don't take lanes on busy streets unless I have one heck of a good reason. As in construction, wrecks, etc
Take lanes, don't take lanes, capitulate to motorists, don't capitulate to motorist, etc
It's all so dang opinionated. Only the OP or someone else in his\her situation would know what is right to do in that particular instance. Passing judgment via a message board is not very accurate in some instances.