You need to define "a difference." Any depth makes a difference at any speed. More depth makes a bigger difference, and more speed makes a bigger difference.

Generally, it takes 40-50mm of depth to be significant. At 20mph, you wouldn't feel it, but could probably see the results in a TT.

At 28mph, one study found going from conventional wheels to a tri-spoke saved 15W of power. That's a fair bit.