I think this is the answer. If you're riding full out, trying to improve, you'll be tired. I'd break it up with an easy day where you ride at a comfortable pace, maybe for a longer distance on flatter terrain.

In any case, that rubber legs feeling is a characteristic effect of lactic acid accumulation in the muscles. It may indicate that your legs are getting stronger, but the rest of you, specifically your heart and lungs are lagging a bit. But that's only one possibility. Try to mix up your training with fast days, long day and hard days, and don't forget to include some easy days.

It might also help to end your rides with something of a cool down cycle, riding easy to sort of work the kinks out the muscles. Pros get massages after rides for this reason, but if you're like me, you need to ride or walk it out post ride.