In my first year of cycling my only cycling was my 17mi RT commute. I hiked on weekends which certainly helps cardio. As I pointed out earlier my average speed during this period doubled.
I decided to join a club and found I had no problem riding the same pace for 40-50mi.
Yes - long rides can help build endurance, but it is not the only way. I do think a bit more than 6mi RT will be needed though.
If the pain is only DOMS (vs. joints, tendons, sharp pain) then enjoy it. Seriously - it's good.
Al