There is no reason not to turn your bars up a couple more degrees. Make small changes and only change one thing at a time, as non-handlebar adjustments can also seem to affect the way the bars feel.

I am curious, however, why you don't want to swap stems? If you feel the problem is too long of a reach, then shorten the reach! Turning the handlebars will mostly move the brake hoods up at first, and the more you turn them the larger the backwards movement becomes, and by that time you will definitely have them turned so far up that they look weird.

So my recommendation is to get a shorter stem, but keep the original one for when you are more comfortable on the bike and can get stretched out more easily... it is common to start the season with your bike set up one way and gradually move towards a slightly different setup - more stretched out, for instance - by the end of the season.