The OP is not alone as so many of us can testify, and have. I smoked for 38 years. Quit smoking, got my old bike back after 17 years from a friend I gave to. Retired, and began riding slowly and surly. I wanted to do the touring I had no time to do as a young adult.

Within three years I had bought my second touring bike and completed my first century ride, fully loaded.

You will be fine, my friend, you will be fine.