You could start worrying about the bottom bracket failing but you might as well worry more about the brakes. You may have fallen but that's not surprising. If you're not going to accept the "possibility" of falling, maybe cycling is not for you.

If you change the BB, you'll probably gain peace of mind but that's about it. You'll just move on to the next component that's likely to fail. IMHO, you'd be better off riding the bike.