Sometimes it seems pretty random when you feel the altitude and when you don't.
Going west, I had more trouble with Togwotee than I did at Hoosier Pass, despite Hoosier being some 2,000' higher. Approaching Hoosier, I had a rest day in Canon City (if you can call it a rest day with a half a day of whitewater rafting), overnighted the next night at Guffey (8,500' or so), and then cruised into Breckenridge with no more trouble that that kind of climbing incurs. Before Togwotee I had a real rest day in Dubois, leaving about 3,000 feet of steady climb to the pass. One might think a week or so over 5,000 feet, including 3 nights over 8,000', would have made Togwotee a comparative walk into the park, but 'twasn't so.