Definitely listen to CDN_Bmxer. I race BMX as well, and honestly, there is nothing more fun than cruising around the track, even when it's mid-summer and you're sweating like a madman. Who cares how old you are, my dad started racing again and he's 40. He got back into it because he was sick of going up to the track and not riding, and partly me telling him he was old. Haha. But anyways, get on a bike and go have fun. That's what it's all about.

As CDN said, the protection is a big factor today. Many many people wear shin/knee pads, and also arm pads. They are much more breathable and comfortable now, while still providing the protection from road rash, shinners, bumps, bruises, etc. Even cheaper helmets are very comfortable and protective. Neck braces are available, and they are very effective in protecting your head/neck area from whiplash and any type of unwanted movement.

Skills are something that will come with riding, so if you go out for the first time and you're slow, give it time. Ask for help. If you are at a decent track with a good environment, I'm sure there will be some fast kids who would be willing to help you out.