OP,
I KNOW exactly what you are dealing with. First time my husband went out with me he cleared 20 miles with ease. It had taken me almost 2 summers to even get that far w/o puking or making it past 5 miles an hour. It was the only time I wanted to strangle him and dump his body in the nearby woods. I'm into my 3rd year and 20 miles max is still all I get at around 10 mph. Mountain biking kills me after the first few hills as my heart rate hits 170 and I'm out of oxygen and my thighs want to explode. I still don't really have an answer as to why I'm pathetically slow and 20 miles is pretty much my max but at least I'm out there.
I haven't had a chance to ride since late December. I had foot problems that needed medication for a couple of weeks. I picked up a really bad cold that wiped me out for 2 almost 3 weeks. I ended up with bursitis after a couple of mile walks and spent a week on a steroid and now have a respiratory infection..probably from the cold and I feel like I'm never going to even get back to 20 miles now.
That being said, DON'T give up. I have to remind myself that one of my bikes (Salsa Mukluk) is 36 lbs and my 200lbs on top of that my legs have to go up hills. My other bike (Salsa Fargo) isn't that much lighter so I'm kind of out of luck on the bike weight department. I had a light carbon bike but I ride highways where the speed limit is 60mph and I was being blown around every time a semi truck went by. I also have to remind myself of where I started..gasping around the block for a month and where I am now at 20 miles and going 10-12mph. Since I can only ride a mountain bike trail once or twice a month, I have to realize my cardio sucks for short hills, turns and twists.
I have both Endomondo and Strava which are great apps and they help keep me focused. I signed up for my first sponsored ride with another lady who also has a fat bike and plan to do a quarter century (sounds way cooler than 25 mile round trip) in June. Now that I have a goal, I feel so much more up to getting a little faster and farther once I get over being sick.
Your wife will probably always be faster than you, but it's all good. If you are up to a little competition, try to keep up (within reason) with her for short bursts. It sounds like you aren't in her way for time/speed and she's happy to wait for you so enjoy the time you spend together.
Good luck on getting a little faster this summer.