I worried for the first few months. My wife still worries, though I think she worries less than she used to. We've negotiated some agreements about when and where I'll ride. The biggest concession is that I don't ride at night if it's raining.
In the end, you both need to be happy with whatever you decide.