When I started riding with the guys at work, we go over a bridge that goes over the intracoastal, so it has to be pretty high to accomidate some large boats. First time we did it, I thought I was going to die, but I did, each way. The first week was painful, the next week was painful, but I was faster, then last week was painful (it always is) and I was much quicker. Also, I have noticed my recovery time is much better, and my legs are not sore for near as long after the bridge. I am now doing the bridge 1.5 times each time out, and this week will be looking for 2 times each trip.
Keep trying and giving it your all, you improve quickly. Good job pushing yourself.