I ride directly to work because I like to get there early. It's also much more of a downhill ride to work than home - Among other terrain, I have two hills where I just hold on and descend as fast as the bike will roll. One is a gravel trail and one is a concrete and blacktop trail.
In the afternoon, I'm more likely to take the long way around a those hills and meander a bit. Consider a capital letter D. Let's say the steep gravel trail that's so fun in the morning is the straight vertical line of the D. The longer, curved part of the letter is a paved neighborhood street with a much more gradual and incline. I'll go that way instead.
There are several "decision points" like this along my five-mile route and I rarely ride home the same way twice in a row.