OP my first rides since mid-2008 were about 3 miles long probably 6mph.
In about a month I worked up to a hilly 6-mile loop that took an hour and I needed a nap afterward.
I did my first bike club ride in another few months. It was very social, 11 miles, a long breakfast, and another 6 miles mostly on the MUP. The route was mostly flat, but the river low water crossing ramps were incredibly steep and I had to stop and rest.
In 8 months I did my first 50 mile ride on fairly hilly terrain. I think I averaged 13 mph.

2 years after starting on the bike, I rode RAGBRAI - a 7 day ride across Iowa over 420+ miles. Some days we had a plenty of hills.
This year I did a tour to Key West, riding 200 miles over 4 days including some days with headwinds the entire time. I rode 70 miles for the Tour de Cure.
That same low-water crossing that killed me on the first club ride - sure, I still downshift for it, but I can still talk while climbing it and it doesn't send my heartrate soaring.
I'm still a horrible climber but I am able to grind out hills that used to require rest breaks.

I was recovering from a major surgery when I started cycling. I had just gotten clearance to ride a bike from my neurosurgeon.
My post-op instructions were to walk a mile every day. My lower body was so deconditioned that just a mile was painful so I just did a mile every other day to give my knees a rest.
My situation was a little different from yours as I had been exercising most of my life and should have laid the foundation for an aerobic engine. Big lungs from swimming, etc.

I guess my message is to not get discouraged and keep plugging away. We've been there. You may improve faster or maybe slower. It's normal.
Give it time. Most of all, keep it fun for yourself.