It's quite common. As long as he is behind you, he's the one to go down if wheels touch. I use these situations as a training opportunity to see who can hang on the longest. Think nothing about it and continue your ride, you'll meet all kinds cycling.