Lots of us have been there. I was 52 when I started with a stationary bike in 2003. I started very slowly, paranoid because I was almost 40 pounds overweight, my father had died of a heart at 54, and his father at 48.
Now, I'm 59, and I rode 7,555 miles this year, including a 241 mile fleche in April, in spite of losing half a lung to lung cancer along the way. Just keep pedaling, and you'll be surprised at what you can do.