For me, not only did I switch from cotton to spandex sports undergarments under my street clothes, but I went with the addition of a really slick leather saddle as well.
I definitely trust you, the switch made a huge difference in after ride comfort.