Yes, many roads are congested.

However, for the last half dozen of my rides, each of over 140 km and starting and finishing in inner Tokyo suburbia, I've completely avoided congestion other than for low-speed (and thus safe) congestion of half a kilometre or so along one stretch of a road named Onekan, and short stretches adding up to a couple of hundred metres in Hashimoto (Sagamihara).

Cycling infrastructure is .... well, it's lip-service to cycling infrastructure. Delivery drivers work long hours for which they are poorly paid, but they (and other drivers) aren't ground down as ruthlessly as they would be in various other nations; and if they do feel they need a lift, stimulants aren't as freely available as elsewhere (and coke is almost unknown). Of course cars and trucks can and do crash into and even kill cyclists, but I'd be interested to read factual evidence for a claim that cycling in Japan is very dangerous compared with cycling elsewhere.