I totally get it. I you fell and got a concussion, that would be a hard thing to not have in your mind. I guess it's the old adage of getting back on your horse. The more you do it the less you will fear it.
I have not had to deal with this with riding yet, but I can totally understand it. I have lyme disease and I struggle with dealing with things that scare me that never would have when I was younger and didn't have health problems. And I've had to go up an down with a few of them.
For example, I play guitar and bass guitar and drums and played in my Church for a long time before I got sick. Then I had to step down from not having any energy from Lyme. Then as I got better I started to play again. I've had to start a stop a few times from health issues and each time I start it's very nerve wracking. At the moment I don't typically struggle playing because I've been fine playing for over a year now. Now it's just business as usual. Now when they ask me to sing a solo, that's a whole other level. I've only done it 3 times in the last 10 years and one was a duet with my wife. But talk about nerve wracking. I don't know if I'll ever get used to doing that.
Anyways... long story short, the more you do it and have success, the easier it will get on the mind. But it's completely understandable to have it stress you out more in the beginning. Since having a bad experience. Reminder, I need to wear a helmet more. That's one of the hard parts of winter. I've been wearing knit hats more and not a helmet......
I need to ask what people do for helmets in the real cold. Like do you have an extra helmet that's larger than normal and were a hood on you head underneath it......