I had tubulars on my track bike the first couple of days that I had it. I threw my chain and totally locked the back wheel - it was completely toast. There was a 1/2" wide 4" long section where the rubber was completely gone. But the silk held and I didn't notice it until I got home (it was a pretty dark trail).
They handle pretty nice though - really great feel, really fast. Maybe you should just get a set for racing?