That's the tough part of watching a grand tour - they really go fast and they're gone... that explains the huge crowds on hills of course but it always amazes me how much time people are willing to give up staking out a decent spot to watch for about 10 seconds of noise and fury as the riders go blazing through.
Sounds like you had a great day!