In general, yeah, a smaller frame is easier to size up than the other way around, but do note that smaller frames also tend to have shorter headtubes and that will limit the height of the handlebars to an extent. If you are not so flexible, this could be an issue. There's a good variety of stems out there so this may not be a problem. Also, I notice lighter performance orientated stems tend to be from 90-140mm. Your options for a lightweight 50-80mm stem is really limited so I guess this makes smaller frames a bit more attractive to someone who is weight conscious.