Track bikes will probably have a higher bottom bracket to avoid striking the banked track which means a higher centre of gravity which means worse handling all else being equal.

Here are some stats for a Cannondale CAAD10 track frame againts the CAAD12 road frame. Track frame numbers first (cm) then the road frame numbers:

Seat tube angle: 75 - 73.8
Head tube angle: 73 - 72.9
BB Height: 27.6 - 26.9
Wheelbase: 99.2 - 98.0
Headtube length: 14 - 13.9
Stack: 53.4 to 55.1
Reach: 40.8 to 38.7

Lack of a rear derailleur hanger may be one issue you have to overcome if you want to race.