Just assert yourself. Take the lane when you come to the stop sign and people will just wait behind you. If you are off to the side and they can almost but not quite get around you, people seem more likely to get frustrated.

Why do you have to wait so much longer for traffic to clear? Do you mean you are at a stop sign and the road you are crossing has no stop? Sometimes at particularly bad roads like that, I will turn right onto the road I'm crossing, and then turn left at the first place I can. In general though, I find that I can pull through that sort of intersection about as fast as most cars do.