Carry a mini-tool, tire levers, pump or CO2 inflator and cartridges, inner tube, water bottle. Wear suitable bicycling clothes, helmet, sun-screen and insect repellant. Chain maintenance: White Lightning wax lube, Simple Green or orange citrus cleaner, a clamp on chain cleaner (Park CM-5 CycloneŽ Chain Scrubber or equivalent). Safety: front and rear lights, helmet, mirror and eye protection.
Optional: rear rack and bags, gloves, rain gear, camera, glueless patch kit (to help other riders with flats).
Practice changing inner tubes, front and rear, at home.
Read bicycling tips on safe riding practices and accident avoidance. Learn bicycle maintenance. If possible, vary your route and explore the areas that you ride through. Find the easiest route, the safest route, the fastest route, the most scenic, etc.
As for the arm discomfort, some shops will help you fit the bike. You might just need to get used to it or try another bike with a different riding position.
Know that you could be an inspiration for others to start riding. Good luck, and check back often.