Go when it's dark - weekend days are a nightmare. Many racers hit the park after nightfall or around dawn, and you can average 20mph laps without too much interference. Just remember that pedestrians always have right of way, no matter how stupid they may be.
Pedestrians have the right of way if they have the light. Crossing on the diagonal without looking is extremely inconsiderate for all user groups (runners, speed skaters, nordic skaters, cyclists, equestrians). Most pedestrians are terrified of cyclists by now, which isn't necessarily a good thing, but most will notice you more easily if you let your freewheel click.
I'll ride central in the evenings on weekdays, never on the weekend. That's what NJ is for. The west side highway is actually more tolerable early in the morning on weekends because most of the slower users you encounter are following traffic laws, instead of just a massive j-walking party. Central park is actually good in the winter months as well.
Look out for horse manure in the southern end, and there's a family of raccoons that live at the base of harlem hill that you don't want to run over.