Should you worry about weight? Let's consider an extreme example - all things being equal, should you choose a 20lb bike or a 40lb bike? Obviously, the 20lb bike will be easier and more pleasant to ride, and even if the 20-pounder costs more, it's probably worth more to you.
The point is this: OF COURSE weight matters, and it should be among the factors you consider. I'm really tired of this idea that weight is a silly or somehow shallow factor to factor into thinking about bikes, or even that you should only concern yourself with it after you've demonstrated some arbitrary level of skill that somehow justifies caring about weight. The reality is that lighter weight makes a bike nicer to ride, period, and easier to carry as well.
But the trouble is that there's no easy answer - weight matters, but the differences aren't as extreme as 20lbs (though even 3lbs feels very different between two bikes), and it's one of many factors you might consider, including geometry, price, components, and so on. I would think about weight, but if you're getting a killer deal and saving money is important to you - that might be worth a weight penalty. For what it's worth, my cross bike - which I race - weighs 23 lbs. Weight is important, but a real bike that I could afford was a higher priority than an imaginary one that weighed 5 lbs less. So obviously there are other factors to consider! But if I had the means, I would go lighter in a heartbeat.