The bits which make go better are improved tyres and toe clips.
On muddy, wet rides, fenders will keep you cleaner and drier.
For long rides, over 1 hr, padded cycling shorts may help.
Any ride too long to walk back, take the neccessary repair kit. I have a short distance kit for punctures only, and a longer distance kit with more tools.
For any ride, helmet and gloves will ensure that a spill is mostly harmless. A waterbottle cage will help you carry enough water.

For some of these upgrades, a $10 solution (toe clips/waterbottle) will work perfectly well, but a $100 solution (clipless pedals/camelback) will work a bit better.