Based on observation, it will slow you down some.

There's not any way to just deduce the answer. What actually happens is that you're moving through the air at a skewed angle. Your airspeed is greater than it would be with still air, your "frontal area" is actually the area on a diagonal, the drag coefficient changes, but no telling how. The force from the wind will be at a skewed angle, also.