I ride a 47 year old Legnano and can attest - components make up a major part of the ride quality.
I had 22mm clinchers on Ambrosio Evolutions on last year (120psi) and switched to NOS Nisi Moncalieri with some cheap tubulars (also 120psi) and respoked to 3-cross from radial with double butted - and changed my headset to a '70s Nuovo Record - went from Cinelli Giro bars to a Nitto Noodle - the bike feels completely different.
I have a feeling that with 80 and 50psi in your tires you're going to be hurting your wheels on the bumpy stuff, unless you have a very wide profile tire. Which kind of negates the purpose of a racing-geometry frame.
It may be time for a dedicated bike for this kind of terrain!
I don't take my racer out on the rough stuff if I can help it.
It's like taking a Ferrari camping - what you really need is a Jeep.