Cool air that you breathe wont hurt you. Very cold dry (less than -15 C) air will cause a kind of a hacking cough just because it really dries out your throat. Also the massive amount of cold air moving past your nose can freeze it.
When extremely cold breathe the warm air inside your first layer.
But at your temp and humidity it should not be a problem.
If you get chilled cause you are sweating and then stop riding this can make you more likely to catch a virus or cold etc.
Manage your heat dont seat too much ventilate.