There's one simple thing you can do to ensure no nasty stuff in your bottles: dry them. Nothing can live and grow without water.

After rinsing out my bottles, I turn them upside down in the dish drying rack. And I never store them with the lid on.