Yup, plastic takes a long time to decompose. Solution? Burn them.
Seriously, I have 3 cheap water bottles that came with the kids cheap walmart bikes, and I've been using them for the last 3 years. The alternative would have been to throw them away, so I figure I'm at zero impact so far.

Yeah OK, if a cyclist uses 30 water bottles in his lifetime, that's about equal to what I've seen some single people at work going through in 2 weeks. I'm not sweating it personally. After all, I could be tilling my entire lawn into garden and growing my own food, and I could be running my computer off solar, and my car off biodiesel. Any one of those would be vastly more impact than a water bottle every few years.