I would say that a cyclist is more affected by a strong crosswind while traveling at higher speeds because the total wind resistance would be greater. You have 1) a perpendicular wind force vector, and 2) a parallel force vector both from moving forward, and from the effective forward force of the crosswind. Of course it's been a while since I've taken physics, so I might be talking out my a$$.
But personal experience on the bike supports this, I think.