Well, yes, your friend is right. Logically, there is a higher risk to riding on busier streets. But you could just as easily not see debris on a bike path, hit it, go flying over your handle bars and break your neck slamming into a tree. I'd venture to say that in any riding situation, the more you pay attention to your surroundings, the safer you'll be.
It's true, drivers don't see things unless they're huge and have flashing lights. Even then, they miss a lot. But if you're smaller, lighter and more manuverable and paying attention, you could most likely get yourself out of trouble without much if any damage. Accidents happen. It's a part of life. But they shouldn't stop you from living your life.