You have to be careful because of notch-failure in aluminium. All cracks start at surface imperfections that cause stress-risers. You'll want to lightly sand the scratches and polish so there are no sharp edges.

It may be late to suggest this, but fietsbob's idea is right on. Easier to have the stem off the bike and thread it over the bar. Rotate the stem so its narrow clamp-section lines up with the tighter inside-bend of the bar and it should slide through easily. Then rotate the stem again to go around the next curve. It typically requires rotating the stem 2-3 times to get it all the way through depending upon the particular curves of the bar.