I live near Central Park at the boundary between the Upper East Side and Harlem. Never had a problem. FWIW, I rode the full Central Park loop around midnight multiple times, no problem.
As dendawg said, the park is closed from 1am to 6am and you can get a ticket for disorderly conduct.