Folks are just pulling your chain

If you are using the bike to commute, your components are fine. When something breaks or wears out, move to a nicer component if you want. If you are racing, any deficiencies will be quickly apparent. As for fit, it does seem you could have used a size smaller but what you do, if anything, depends on the circumstances of your purchase. If you just bought it and can return it/ exchange it with no financial loss, then that might be the way to go. If there is no return and you will lose money trading it, I'd ride it as is if you are comfortable. There are nearly as many opinions about proper bike fit as there are cyclists.