There are horses in NYC: horse-drawn carriages and mounted cops, mostly. People ride horses in Central Park (or used to -- did I hear the stable closed a couple years ago? Is there another?). But I think it's perfectly legal to ride a horse on the street in NYC. Not sure which bridges you can legally ride your horse over to get into Manhattan, though.

EDIT:
http://www.midmanhattan.com/horseback.htm