That's 15-20 hours a week. Some amateur racers do that, albeit not all year round. But they're also riding harder for much of that than you would be.
It's probably not too much once you are used to it. But that may take a while. You'll probably need to take a break (1 day to 1 week) when you get too fatigued. You'll also want to arrange your schedule so that you're fresh for some things, perhaps the martial arts practice.