Yeah, I have had those thoughts too - but it is better to live the life you have then try to never get hurt. Life is not only about duration, but quality too. You are out doing things you like and you are taking the risks that you feel are acceptable. Change for the future if they no longer are, but don't go worrying too much about the decisions you made. You have plenty of other decisions that you are currently making to worry about!
Just like sharks, what gets you is normally what you never see coming.