Do you have actual problems with your vestibular system, or are you just full of nerves, if you have actual balance problems I would not ride a bike.

I used to teach martial arts and it was very discouraging to have to tell some people that they just couldn't do it (I'm talking about disabled people), even though they were motivated.

There are limits to everything.

If you have no vestibular system problems then stick with it slowly until the nerves go away.