If your lights blind oncoming traffic and cause other vehicles to crash, that's a disadvantage. If your lights cause you to be pulled over by the police, that's a disadvantage. I don't believe 4000 lumens are necessarily problematic, but I can think of plenty of scenarios in which significantly exceeding the output of vehicles around you could produce negative results for you or others around you. Even at a much smaller level, there are certain situations where I don't run my P7 at night. Almost all of these situations have to do with not wanting to stand out (at least until passing certain areas).
Cars typically run 55 W halogens. That's roughly 1500 lumens per lamp and they have 2...sometimes 4 with driving/fog lamps. I have one more then most and one less then a few. My lamps aren't too much higher than a sedan's lights and are lower than a SUV's lights. I also ride near or to the right of the right wheel track of vehicles. Since most cars have the right side light set higher, I'm not that much different from a car without the driver's headlamp. I doubt, highly, that my lights will blind oncoming traffic any more than a typical car would.
As for riding in dicey areas without light, that's not for me. I run them on full with the helmet light. If people think you are a car, they are less likely to jump out and grab you. If they do try anything funny, 1500 lumens of light in their face will blind them long enough for me to get away from them.
But, from a practical standpoint, I'm not worried about people. I worry more about the cars.