My experience is that earlier is better than later. I find the quietest time of the week (in terms of traffic) is Sunday mornings. So -- all things being equal -- my very favorite time to ride is leaving at dawn on Sunday and getting as much riding done before the "Sunday drivers" emerge early in the afternoon.
A second favorite time to ride is Saturday morning.
On Saturday afternoons the roads fill up with people rushing around doing errands. On Sunday afternoons the crazies are out, throwing coins at CliftonGK1, and other garbage like that.
There is a century ride on Saturday morning that doesn't start until 8 a.m. I just realized this morning I think that's a crazy-late time to start a century on a beautiful summer morning. I may either leave from home and ride to the start or just show up at the starting line in my car sometime between 6 and 7 and take off on my own.