What is the advantage of a race bike being twitchy?
All I know is, I get shaky going from the hoods to the drops (especial on a descent)

They're quick handling, which is their advantage. They shouldn't be twitchy at speed, hands or no hands.

Quicker response to steering inputs and more response to steering inputs.

Besides stiffness of the frame & fork, the wheels, rake of the fork and wheel base; are there any other factors influencing responsiveness?

Head tube angle, wheel base, rider fore-aft position, stem length, rider height, frame size, tire size, tire pressure...

When you switch from the hoods to the drops, all you're really doing is flipping your thumbs under the handlebars. You don't really have to take your hands off the bars to do it.

A bike with relaxed geometry will still be twitchin' if the rider isn't relaxed, too.

Thanks for the technique tip.
I will try it after work (if it does not snow)