It's funny ... if the OP said "some Italian" or "some Irish person" it would seem objectionable .... but "some woman" isn't sexist?
I don't care. This is a very mild distinction, and not (to me) offensive. But it is all and only about some archaic idea that men are supposed to be physically superior to women. But really, all it is, is this guy saying "I am sexist."
Not worth a debate. If he gets all angst-ridden over being passed by a woman rider, that is his issue.
Some folks get upset if certain types or "classes" or riders pass them, others, different groups. Some folks can't stand being passed at all ... and some don't care if they are overtaken or not.
Not an Earth-shaking issue, in my view.