In scheme of things, drinking coffee is much less harmful -- and has a number of benefits -- compared to other habits such as smoking, drinking alcohol, taking drugs, etc. Unless you live like a monk and only drink water, coffee is preferable in many respects to many other non-alcoholic drinks such as sodas, fruit juices, milk, shakes and many teas -- particular if you drink coffee without sugar.
If you don't like the taste of coffee and/or have never developed a taste, it's no big loss. However, I get a lot of enjoyment out of drinking coffee and have no qualms about drinking 3-4 cups a day.