^ i didn't experience any harshness, in fact the 28mm conti grand prix 4-seasons i put on the bike are quite a bit plusher than the 23mm vittoria rubino pro's on my road bike, and even easier riding than the 32mm conti gatorskins on my hybrid.
but yes, the handling with rack and panniers is very unusual and will take some getting used to. and for the times when i still want a zippy agile ride, i can always just throw my backpack on and ride my titanium road rocket into the office when the weather cooperates.