Yes, there's no doubt that my 20-spoke wheels are the actual "weakest link" of my bike (in dealing with my 240 lbs). (In partial defense, I am 6'5"... )

That said, I've had no issues with my wheels in two years (Mavic Aksiums), despite only 20 spokes. They stay true, and the only slightly negative thing I've experienced has been a bit of occasional spoke "ping" when I am really mashing it in a low gear, going up a steep hill, which is only partly a weight issue. Granted, they're not the lightest wheels around.... But i can't complain about their ruggedness.