Simple way to not only have the water taste ok, but to clean bottle. I squirt a little toothpaste in bottle about once every week(about as much as you use to brush teeth). Add water and put lid on and swill it around for a bit then dump it out and rinse it out good with water.
Always when not in use, leave it empty and lid off. Works great for me.