I'm just a rec rider--no interest in racing or crushing any souls if I can help it. My goal is to hit the road next spring as strong or stronger than I was. I plan to train a little every day and ride when I can.
I'm in a similar situation. I'd set a more specific goal than being a "stronger rider", and work toward that goal. In order to work toward that goal, you'll have to do some research and find the workouts that support your goal. I'd suggest checking amazon.com for books by Joe Friel and/or Chris Carmichael. IMO, Carmichael does a better job of addressing the needs of non-racers.
Living in the Adirondacks, I'd personally be inclined (no pun intended!) to work on my climbing, but I'm not you.
Just pedaling aimlessly will help somewhat, but being focused yields better results - even for us non-racing recreational riders.