No complaints about my Polar 24 oz insulated bottle, which Polar says is made from LDPE 4. I dislike that plasticky odor and taste, but don't notice any problems with the Polar bottle.
But if I was very sensitive to any plasticky odor but was okay with disposable clear plastic bottles, I'd just find a way to make those work for me. There are disposable clear bottles of water that are a close fit for standard bottle cages, and a little tape or foam would take up any slack. You can retrofit pull-push squeeze dispenser tops. Or poke a hole in the lid with an ice pick, or drill a hole, then tape over it, wrap Saran wrap around the top secured with a rubber band, or plug it with something until you're ready to drink, then stuff the tape or plug in a pocket.