I've ridden this path on several occasions and am amused at our difference of perspective.
#1- The shade. A tree lined bike path for some is a pleasure. It's relief from the hot sun, shelter from the wind and not too tough on the eyes.
#2- Lots of walkers. Yes, on a beautiful Sunday in the midsummer it does have lots of walkers. It also has lots of families on bicycles and sometimes solo riders on road bikes and each must accommodate the presence of the other. While the walkers seem to be in your way you and your family may have seemed in the way of some other user of the path. It's called a Multi-Use-Path or Shared-Use-Path for a reason. If you ride it, expect to have to share it. Or as you point out don't ride it. Or ride it at odd hours when it might not be so crowded (that's my choice- I avoid trails like this at peak season and times)
#3- Road crossings. Yep. There are. Tunnels and bridges over the roads are cost prohibitive and so rail trails often have to cross streets. I've noticed that most car drivers are pretty accommodating and stop (especially for riders in groups and families with little ones). But if you're in a hurry and prefer not to stop you could try riding the road. You'll still be obligated to follow the rules of the road and stop at stop signs and signals but there might be fewer of them. I would be curious to look at a road map of the trail and compare it to an on-street route to determine, which might require more stops and have more intersections.
#4- Uneven, bumpy conditions are, unfortunately, a fact of life on many MUP's over 10 years of age. Unlike roads, where the weight of heavy vehicles pound down most tree roots those same trees that line the road spread their roots under the pavement and push upward and create bumps and frost heaves.
A solution you might like is to cut all the trees down. Then riders would be exposed to the full sun to better see the path and the wind, when it is at your back especially, would be unimpeded. I don't know if all users of the path would prefer this solution, however, and might be inclined to cycle elsewhere but that would leave it more free for you and your family.