Never banged a fork against a work bench but my experience is as follows...

I have three bikes, all with reputations for quality:
All CF road bike (18 pounds - 23mm tires),
Aluminum road bike with CF Fork (23 pounds - 25mm tires)
All aluminum (commuter) with aluminum fork (34 pounds - 32mm tires).

Running (as close as I can) 75 pounds in the front tires and 85 pounds in the rear tires over the same terrain, The all CF bike is the smoothest, followed closely by the aluminum bike with CF fork and, easily the harshest ride is the aluminum bike with aluminum fork.