From doing 400km and 600km brevets, I really don't think riding late at night is that much of a problem. It may be better if you can find a group of similarly crazy people to ride with. About 6 am, a batch of sleepy people decide to waste gas and it isn't as much fun any more.