I think its more about where you ride and how you ride (and also how the wheels were built) that makes a difference.

Does it bug me? no.... it annoys me but then I just remember to be more careful where and how I ride (no jumping off curbs, watch out of pot holes etc). If it happens a lot, I consider getting a stronger wheel.