Average power doesn't mean it will feel the same. As some mentioned, the bike slows more between each pedal stroke when fighting a headwind, or climbing a hill. So, the average power may be the same, but the peak power (force) during the pedal strokes will be higher when riding against a larger force (wind or hill).

I also agree, that head or cross winds, can generally just be more fatiguing. After all, we are still balancing on two wheels and the effects of head and cross winds makes that a little bit harder. Or sometimes a lot harder.