drag is proportional to the speed of the fluid (air) moving over the object (you). With wind at perpendicular angle, you will face a higher wind current than if you were just riding with no wind, in which case the only wind moving past you is in the direction you're facing.

For instance, if you are facing a wind that is as fast as your bike speed, you can add the perpendicular wind vectors together, and the resulting drag would be sqrt(2) the amount you would face if there was no perpendicular wind.