I no longer try to max out fast downhills, it's not worth it to me. Shallower ones, sure -- 32 mph instead of 27? Yeah! I can reach the highest heart rate of a ride on these sometimes. Go, go, go!

Your KOM downhill sounds safer than many others, with good sight lines, no side roads or driveways, smooth pavement.
But I've pinch flatted on a nice newly paved downhill, on that one big piece of gravel that was kicked out from a driveway. I didn't see it. At slow speeds, I would have bounced over it instead of flatting.
No deer or dogs to run out suddenly? Or even squirrels.
I do let myself go faster on roads like the Blue Ridge Parkway with swooping curves, good sight lines, no drivers pulling out, and few animals darting out.

Stopping distances go way up.
One thing that got me to slow down on steep downhills: Stopping distances are extremely long, and it takes a while to burn off even the first portion of speed.

This math puzzle applies to bikes as well as cars. There's a limit to how much braking force you can put into a bike before the rear tire starts lifting off the ground. Just like a car has a maximum braking force on it's tires.
https://www.roadandtrack.com/new-car...mph-heres-why/

The question in the embedded youtube video is: (No, I sure didn't guess the answer.)
A car is going 70 mph as another identical car passes at 100 mph.
There's a sudden obstruction ahead, so both start maximum braking right then.

The 70 mph car just barely stops before the barrier.
How fast is the 100 mph car going when it hits the barrier?

~~~~~~~~~~

Stopping time is related to kinetic energy from the speed & mass, which increases by the square of the speed.
This would apply to a 30 mph cyclist vs a 40 mph (with 78% more kinetic energy) or 50 mph cyclist (178% more than 30 mph) too. With wind resistance to help slow down, the faster speeds wouldn't be "quite" this much worse.
After the video, I started noticing how much easier it is to slow from 25-30 mph compared to 35-40 mph.