Had a problem, and found a solution. So I'm posting this because I'm sure someone will have the same problem and would be interested in how to fix it.

So I decided to wash my bike. I took everything off that I could and gave it a good wash. When I was putting the bottle cages back on I got a little too enthusiastic and I snapped one of the bolts holding on the bottle cage. Stupid, I know. The bolt was snapped off deep inside the hole and so I started looking on the net for solutions. I found advice like buying special drill bits, drilling holes, and re-tapping threads. I think this advice is rather generic and would work well on something like automotive bolts which are very hard.

But the bolts for a bottle cage are rather soft in comparison. So what I did was I got a cheap jewelers screw driver and gave it a few taps with a small hammer to drive it a tiny amount into the exposed face of the bolt. Then I was able to use this to loosen the remains of the bolt just enought to be able to move it with a pair of needle nose tweezers.

I ruined the tweezers but got the bolt out. Now everything is fixed and I am happy.

I'm sure someone else has either done this, or will soon. Just thought someone might like to know how to fix this.

Of course I have learned that bottle cage bolts only need to be finger tight, not gorrilla tight.

Thanks for the tip. Was the jewelers screwdriver a flat head or a philips(star)?

I used a flat head jewelers screw driver.

Aluminium bolt.

Actually that is partially incorrect. While it's true you don't want to make the bolts gorilla tight, but finger tight will eventually loosen up the cages due to road vibration, and then you'll have loose cages and a noise; this loosening up will probably happen on the very first ride! Even snug cage bolts have been known to come loose after several rides, but finger snug will loosen in the first ride...unless you have gorilla strength hands!!

Yeah, about as much torque as you can easily apply through the short end of an allen key with your thumb and forefinger should do it.

If you use a drop of thread lock [ala LocTite Blue]
then you don't have to torque the bolt so tight to keep it there..

bottle cages are different loads than the bottom of the rear rack.

IMO it's best to avoid threadlocker when possible, for a few reasons: you'll learn more about appropriate torque for a start, you won't have to keep buying the damn stuff, IME you never need it unless something's wrong*, and it's nasty stuff; how's the stench on it!

*Even on applications where it's de rigeur like RC cars, I've found that most fasteners just don't require it (which you can only ascertain by eschewing it); the secret is simply to go for that sweet spot where it's plenty tight enough, but not enough to cause significant damage or wear. Those who recall the recent thread about torque wrenches prolly know what I'm talking about. Although getting back to threadlocker, obviously there are some applications where it's the perfect thing.

Heck you could just use a thin lock washer like a split washer if you continue to have loosening problems, then snug it down with a allen key with the thumb and forefinger as Kimmo pointed out and the spit washer will keep it from loosening. A star washer will work well too if the bike's frame is aluminum, the serrated edges of the washer will bit a bit into the bottle boss and into the bottle cage frame; but if the cage or the frame is carbon fiber or steel I would use the split washer. This would work a lot better then threadlock fluids.

OP ruined tweezers and probably a screwdriver.
A counter clockwise(anti clockwise) drill bit
could be used over and over. Oh, and you can
also use it to make holes(tiny ones)

I'd rather ruin a cheap pair of tweezers and a cheap screw driver than ruin a bike frame because I managed to put a hole in the wrong spot. I'm sure others would be competant enough to use the drill method, but I'm confident something could go wrong for me.

Understood. I only mentioned it as another option. Maybe others don't have a tweezers
and small screwdriver handy. Of course one needs a drill to use the drill bit option.

I don't think I'd consider using a mains or even a cordless drill for that job; that's one for the dremel.