Agreed, the fact that they can actually gain any advantage by drafting on uphill stretches lets me know that I have never been at their level.
Because you always see them around fast cyclists, it is easy to miss how fast they are... If they were to pass me, the difference in speed would be obvious... In fact, you would be amazed that I could stay so relatively balanced at such a slow speed.