Hmm, ok...

There are handlebars that sweep back even more than that, so as far as the sweep goes, you probably won't have any problems at all.

What I might get concerned about is whether you've introduced some droop to the ends. That can put your wrists at an uncomfortable angle. I wouldn't ride like that for very long, if at all.

Right now, I think you have two options: get a shorter stem, or get a smaller bike.