Hello DanBell. Tokyo is a great city but I never rode there. I have ridden in Mishima City which offered more varied terrain. It was a blast!

What stem are you using now, quill or threadless (I believe it comes stock with threadless but don't know if you changed it)? A quill stem extended out a bit will flex more. Also, are you using bullhorns? Of course, these will flex some when climbing on the horns. I use a quill stem but also use a 57cm flatbar. It is nice and stiff.