Hey, I prefer headwinds, buddy!

Personally you need to weigh up the pros and cons of the issue for yourself. The key issue is how much you get out of riding. The fact is that every activity carries risk (even being a couch potato), and cycling is actually a much safer activity than many people perceive it to be. In truth, you are probably more likely to be killed driving than cycling.

Having said that, if you feel nervous every time you ride, it mightn't be a particularly safe activity for you. Fear is good in certain quantities, but being excessively fearful is likely to have a negative impact on your safety levels. Basically, it's no good going out with the "I'm gonna die today" attitude.

I support what the others have said, start slowly and see how it feels. Build it up gradually, this should help you get your confidence back. Basically take your time and see if you rediscover the pleasure of riding.