Chatting with my sister yesterday, she was talking about how bad she felt, and everywhere she was sore at the end of a 32-mi bike ride (her personal best).

I do this distance three times a week and more, and of course for me getting off the bike, shoulders and neck completely relaxed, it's
ahhh, this is the way I'm supposed to feel

If it's not working for you, a few things to concentrate on. Use your hands for balance, but not for support (except of course in braking).
Support yourself with core muscles, think about keeping your shoulders down and relaxed, and your neck relaxed, always keep your elbows bent and your wrists straight.