Now that you mention it ... yeah, I've done that before now. More recently I use a cold chisel as a wedge/lever, but it can get in the way. Don't use something that can slip, like a screwdriver, because you're trying not to scratch the handlebar.