If you're worried about the taste of the water, you're not thirsty enough.
Jeez, what a bunch of... ah, shutting up now.

I have been using the same cheap plastic bottle that came for free on a cheap bike for about 4 years now. It doesn't look any different than when new other than a bit worn on the outside.

I also reuse "disposable" bottles for weeks at a time. What's the problem supposed to be with this? I think people are so used to sterilized everything that their bodies react badly to anything vaguely stale.