It sounds to me like you would benefit from doing some longer rides. Two 30 mile rides in one day is not the same as doing a 60 mile ride. If all your rides are 30 miles or less, you'll likely begin to feel the accumulated miles on much longer rides. After you do some 60-70 miles rides you'll begin to get used to that distance. If you go with a group, even better if you need to sit in some. As mentioned, try not to focus on average speed. Spend some time researching interval training and put your energy there if you want to get faster.

You will see a variety of opinions when it comes to lifting. The extra weight and bulk it adds will hurt you on the bike. However, I agree that cycling alone is not enough for me. I was doing some floor exercises, minor lifting, etc. to focus on the core. At the beginning of the year I joined a gym and have enjoyed the variety of equipment that allows you to focus on certain areas. My main interest is to strengthen the core, but will also spend some time on the legs, arm, chest with specific equipment. I also try to do some cardio if weather canceled a ride.

If you're not already doing so, I'd suggest you seek out a group to ride with, since you will gain some additional skill and knowledge from those you ride with.