The size of the frame has very little to do with its rigidity or riding feel. It is the design and implementation that define those characteristics. The disadvantage to stem and seatpost adjustments to make large fit differences is that these longer levers put greater strain on the stress areas. And the longer levers might be less rigid (in a noticeable way)... BTW, this is one reason that I prefer the older threaded fork style over the modern threadless. In the older style much larger range of adjustments were possible with handlebar height without resorting to 'extender's which I don't believe work very well.
If going this way, select the longest seatpost to ensure that as much as possible can remain inside the seat tube (well beyond the min insert level) and try to get a bike with a larger diameter seat tube.
Given a good frame and appropriate extenders for stem height, length and a long sturdy seatpost, the comfort should be 'the same' as if the same frame were in a larger size. But only if the extensions do not compromise the function, which they can depending upon their design.