Depending on the cast, you probably won't be able to grip the bars or hood well enough. I broke my scaphoid inline skating and had to have surgery since it wasn't healing. I was off the bike for a long time. So, I inline skated since the cast was also a nice wrist guard.
Hope you heal fast!