That's usually true, and not an illusion for most people in most places. A decent rider maintains speeds between 15 and 20mph or higher. 15mph is pretty decent wind on most days, so you're usually riding faster than the wind, meaning even with a tail wind, you're riding into a head wind.

Add to that the reality that you'll ride faster with a tailwind. As a very rough guestimate, you'll probably ride at least fast enough to produce a net headwind of 10mph (if you have the gearing, because that's what it takes for the effort to feel normal.

So headwind or tailwind, you'll alwaus be riding into the wind. The difference is the gear you use, and your ground speed.