I have exactly one place on my route where I can take a back street, for 2 blocks. On the way in, in the morning, it's two right turns and a stop sign. If the stop lights are running (they're blinking yellow until 6AM) I take it. If they're blinking yellow I go the main road to avoid the stop sign.

On the way home, it's two lefts across the main road. If it's before about 4PM, I'll usually take the back road because I can get back onto the road again. But if it's after 4 usually, definitely if it's after 5, it can take quite a long time to get back into traffic on the main road again at the end of the detour, and I would have been better off staying on the main road. I can definitely keep up with traffic there, and in fact I can keep my speed way up higher since it's a gradual descent if I stay on the main road.