Try loosening the expander bolt. Note that you need to turn it clockwise to loosen, opposite normal convention. You need to remove the levers to do this.