If you live where there are no hills (eg. tablelands, prairie, etc) wind is a special opportunity to get some decent climb simulations in.
Just how hard is too hard is pretty subjective. It'll depend on your strength, fitness, skill, and confidence.
Riding in the wind is much more difficult if it's gusty (see edp's post, above). Gusty wind saps your physical and mental strength faster.
Go out and try riding in wind of various strengths and have fun working out your own limit