I know what you are talking about, some bottles do this, some don't. If you have left the water in over night you should wash the bottle any way. When it is dry that tase/smell goes away. Don't use a bottle that has beem left overnight if you can help it.
The cancer thing you mention was from the news. It is about washing and reusing the bottled water bottles that come from the store with water in them. They are not made to be re used. It was not about bottles made to be reusable bike bottles.
If you have the taste problem a lot, try buying some better quality bottles at a bike shop.