Unless you quit your job, or move your house, you're breathing the same air you would anyway in a car. Most cars don't have much in the way of air filters for passenger cabin air. So you're getting the health benefit of exercise, and the same health detriment from bad air that you'd get anyway.

Additionally, better general health (from exercise) improves your body's ability to deal with problems like dirty air.

If you removed all cancer-causing substances from the world, civilization would collapse, and we'd all be wandering around in the sun eating berries and getting skin cancer.

You can never just say "Oh this is bad ok quit using it" if it's something that is in widespread use. You have to come up with a substitute that works well enough you can switch to it, and give people time to make the switch. Sometimes that's very hard... Asbestos, for example, was a "wonder-material". It was awesome for hundreds of applications. Of course, it's also very bad for you, it turns out. But until we were able to come up with alternatives, the best we could do is say OK when you deal with it, you must take lots of precautions.

For example, your bicycle was produced and is maintained using many harmful and/or carcinogenic substances. All petroleum-based greases are bad for people, the fumes produced when welding are carcinogenic, etc. Ideally we'll eventually have substitutes that work as well or better without the harmful side effects, but I don't think you want to stop riding your bike until we do.