I used to think that, but then I rode a 700x35 Kenda Kross Supreme and found out it is not univerally true.

This is an old cross tire first developed in 1998, and it is now out of production but available cheap through various sources. It is really terrible on paved surfaces and hardpack. The tread is open for mud and loamy soil, and the blocks are very squirmy. You can feel them moving around when you turn whether high or low speed.

And this is at 85psi, the max rating on the tire. It's worse at lower pressure.

If you don't have a knobby, though, I would think that few tires would squirm at high pressure.