I biked up Mont Ventoux from Sault then descended to Malaucène. I would think that you'd have to walk from one of the towns at the base of the mountain. The only parking area I recall is at the cafe/restaurant 6 km from the top on the Sault side, and I doubt that race officials would allow parking there on race day. Anyway, if they're starting from Malaucène, you'd need to be on the other side of the mountain.

I once saw the Tour by accident in the French Alps. People parked in both lanes on the side roads which intersected with the Tour route. I was touring. The cops let me ride on the route because the racers were a couple of hours away. I stopped at the intersection where I planned on turning away from the Tour route. There were people standing along the roads hours before the racers came through. People brought picnic materials and walked considerable distances to find a good spot. After the peleton came through, I pedaled through the parked cars on the side road, and since nobody could drive away until the 1st car drove away, I had the road to myself for an entire hour. Nonstop traffic after that, however.