I was gonna say Richmond too but since I also road race and know there is a big road race culture down there with expensive bikes I guess that is out for you even though they really are a bunch of good guys and girls.
You might like Ashville NC because it's full of hipsters(am I aloud to say that word anymore) Ok I mean it's full of H-people who frown on carbon fiber. But you will most certainly have to deal with hills, big ones.
Otherwise I vote For Providence.