So that would mean I'd have to replace the whole thing?

I'm thinking of putting a bit taller and longer stem and upside down north road bars on it. Would that look silly? I kind of like the way north road bars look inverted. I think the stem would have to be really high for upside down bars to be in a comfortable position though.