Scroll about half way down to SIZING GUIDE AND GEOMETRY
https://www.giant-bicycles.com/us/roam-3-disc-2018
They say either might work. But they also show that the Large of that particular model and year bike will fit best. But that's still a guess. If you can try it on and ride long enough to tell then you'll know something more than we can tell you.
I usually say that smaller frames will feel sportier and larger frames more like a four door luxury sedan. I like sporty. But I only have experience on road bikes with drops.
Adding a longer stem is an option if in every other way you felt good on it. How was your saddle height? Enough seat post in the tube if you needed to go higher?