I was just thinking of doing something similiar. I was wanting to take a mm or two out of the forkends to straigten out the geometry of my bike (make it a degree or two steeper).
Ive been thinking about it for a while, and with only taking that little bit off and doing it well, I couldnt really see any danger in it.
The only thing I wouldnt like is that it would make my toptube appear sloped if you looked close enough when really it wouldnt be all that sloped enough to notice at a glance.