The "will they break" thing is always a possibility but, in my mind, if there is no evidence of bending, over clamping or gouging, then I feel OK with using the handlebar.

Here's a question for all interested in this thread. Is a swagged handlebar stronger than an unswagged one? The swag, if that is what it is called, looks like this...