Interval training and/or hill repeats!
I have had the same problems and these helped tremendously!
My normal weekday ride is a 32 mile loop 3-4 times during the week, with a 60+ mile ride usually on Saturday morning. I replaced one of my weekday rides with hill repeats. This is by far the most difficult day of my week and I have to force myself to do it, but it has made all the difference in my average speed, sprinting, and the problem of fatigue early in rides.