I am also a type 2 diabetic, but I do use insulin. My fasting numbers in the morning are usually 70-110, unless I eat something I should not have the night before. My post ride readings are always in the 180's, but they drop pretty quick. Sometimes I give myself a very small dose of insulin but usually within a few hours its back down even if I don't.
When you ride your body releases glucose for energy, thus increasing the glucose in your blood. Overall, however, your A1C should be much lower when you are active then when you are not. I started riding about 10 weeks ago and I was using HUGE amounts of insulin to keep my sugars in check, I was using upwards of 300units a day. Within a month of starting being active again I was using 30units a day and now I am down to 15ish, of course depending on what I eat.
I am not sure what you mean when you say your only eating plant based foods. Keeping your sugar down really is simple, but not easy. Eat as few as carbs as you can (this is VERY hard for me). Eat reasonable amounts of protein, I eat chicken, pork and beef. If you eat too much protein your body will turn it back into glucose. I also eat good fats, especially if I am going to eat something that contains sugar. For example I will eat some peanut butter with an apple, the fat in the peanut butter slows the intake of the sugar from the apple and help keep your numbers from spiking.
If you are calling 50 miles a short ride you are obviously in pretty good shape already and you might really need insulin to keep things under control. Eventually after having diabetes for periods of time, this can vary greatly between people, your body may stop making its own insulin all together or not produce enough. When that happens you need insulin, medicines will no longer control your glucose level well enough on its own. Have you seen an endocrinologist? They do more tests and give give better advice then a primary doctor.