Just realize, your wife lets your ride a bike without making you feel like ****. You should be happy.
Find her a friend who can enjoy her pace and go do your own pace, next time. Finish your first loop with the pack and join them on your second loop.