Could also have mechanical issues. Bent fork, wheel in dropout crooked, bad headset bearings. There's probably many more possible causes. If it just happens momentarily and at random, it's probably the component called 'the rider.'

Regarding cycling as exercise, you can get out of it whatever you put into it. Many years ago, Bicycling magazine published a chart of calorie burn based on rider weight and average speed, based on putting a rider on a trainer and measuring O2 usage. Of course it was minimal Calorie burn at 5 mph, but at 18-20 it became significant, like 800+ Calories per hour. Which, if that's at 18 mph, is still only 44 Cal/mi. Overall, I think walking at a brisk pace is still the best exercise for weight loss. And yes it must be done in conjunction with diet.