Jeez ... I've been avoiding this thread, cuz I thought it was about rider weight. Oy.
I'm in general agreement with what's been said here. If you want to seriously reduce weight on your bike, you're probably best off getting a new one. Cutting significant weight off of an existing bike is expensive.
I also agree that if you still want to do it, start with wheels. Then you can move on to seats, seatposts, skewers (good suggestion, BTW) and the like.
How big a deal is the weight of the bike? The engineer in me says forget about it. Lose a few pounds on yourself instead. For pretty much ALL of us, there is a lot more weight savings to be had on ourselves than there is on the bike. And a little bit of weight doesn't make that much difference.
The rider in me says ... well ... I dunno. It may be the placebo effect, but to my mind, I tire less quickly and can go longer and faster on a lighter bike (provided, of course, that is equal in other respects, such as pedaling stiffness).
I've heard all kinds of explanations for this ... most of which revolve around the fact that we aren't just point masses dutifully chugging up the hill at a constant speed and direction. We make zillions of little steering and balance corrections, and the torque we apply to the bottom bracket is not constant. Since we aren't going a constant speed, but are in fact constantly accelerating and decelerating, differences in mass are magnified over what you might expect, and a little weight goes a long way.
If my kids were still doing science fairs, I'd cajole them into doing a study on this.
__________________
Proud parent of a happy inner child ...