If you want to improve, you need to ride regularly. Your fitness degrades every week you don't ride at least once. If you want to improve, you have to ride more; preferably 2 or 3 times every week. Try increasing your mileage by around 10% every week until you meet your goal. Once you start riding regularly, you will not be sore after every ride. Soreness mostly happens because you're sitting around too much in between rides.

My neighbor recently celebrated his 70th birthday by going on a 70 mile bike ride (and not a flat one).