Is this a recessed brake caliper nut or bolt? Sounds like you're talking about the recessed nut that fits on the center of the seat stay tube. If this is correct, then consider the following steps:
1. Use some form of penetrating oil (i.e. Liquid Wrench, etc) and let it sit for awhile (try not to get any on the tire and rim, and be sure to clean the rim before use if you do).
2. If you can, carefully try to twist to the exposed part of the recessed nut with a pair of pliers in order to loosen it.
3. Thread a sturdy bolt through the other side, like you did before, and try tapping it out.
4. If that fails, place a 1/4 inch + thick stack of washers that fit over the outside of the nut (on the seat tube side) and then place a smaller holed washer on top of them. Next thread a suitable bolt through the washers and pull the nut out by tightening the bolt.
Note: I once had to drill out a recessed caliper nut frame hole "by hand" with very short drill bit (due to the lack of clearance between the seat stay tube bridge and seat tube). I used a small pair of vise grips in order to hold on to the drill bit). It was a pain, but it worked. You might, as a last resort, consider doing the same in order to drill-out the nut entirely.