File them down. It might take a little bit of effort but it'll work fine. You might want to cover the surrounding area with some old bar wrap or something similar so that you don't accidentally file any of your paint off if you happen to slip. If your frame's black, you can pretty easily rattlecan the filed down area if you don't want to keep it bare.