They do make stainless bottles with old fashion cork:
https://www.ebay.com/itm/VINTAGE-BIC...-/123839530247

If you are worried about BPA type plasticizer - that is a coating on the inside of the plastic bottle. Wash your bottle through your dishwasher a few rounds, and that coating will wash off.