I check my mirror more often on the MUP, often every few seconds. That's where I'm most likely to experience a close pass by someone I didn't hear coming. There are always a few guys using the MUP as their personal race track, threading between slower cyclists and pedestrians on a 4' wide sidewalk, rarely announcing or announcing as they're passing rather than before.
I don't know how often I've slowed to pass a pedestrian safely and courteously, only to find a wannabe racer threading the needle around or between us at 20 mph. I rarely encounter drivers who are as reckless as a handful of cyclists on the MUP. They don't represent the majority of MUP users by any means, but most of us would be outraged if we experienced that same behavior from drivers on the streets.
It's the main reason I avoid using the MUP during peak hours, or with groups.
And on the streets at night, as long as drivers are using their lights, I can tell whether they're behind me or off to the side by the appearance of my own shadow. Don't even need to check the mirror. And if they aren't using their headlights I won't see them in the mirror at night anyway.