I keep my speed down. I'm a morning person, but if I'm riding in the dark, odds are it is past my bedtime. Exhaustion is one of the main causes of accidents, so I go slower. And given the routes I ride most often at night, it really would be an accident. The most likely way for me to go down is me doing something stupid by mistake.
If I ever get around to doing dark rides in the morning, I'd keep my speed down a bit for the opposite reason. A lot of other road users are night owls, and they're sloppier in the morning. I don't trust them to keep me safe.