For me, the most noticable difference is actually the handling. The lighter bike is more responsive to changes in direction and attitude in turns or when I need to shift my bike from one line to another. It also is easier to "unweight" over obstacles or when shifting the overall weight transfer either forward or rearward. The advantages in acceleration and climbing are also noticable, but "flicking" around a 35lb. bike underneath you on a rutted trail at speed vs. a 20lb. bike is very noticable whether you weigh 100lb. or 250lb. Same goes for a road bike.

Reducing rotating mass by running a lighter wheel/tire combination increases the advantage in accelleration and to a lesser degree handling, and it is probably the best place to start a weight reduction plan, but reducing overall weight will improve handling regardless of your weight or experience. I believe this is why you hear riders talking about improved "feel" of lighter bikes.