Good luck with your experiments. I've been down the same road as you and discovered I could ride smaller frames than what I was riding at the time. Over time I've gone to smaller frames and the comfort is measurably better for me. But my riding style is longer rides at a fairly constant pace/HR. Also, don't forget you can move your brifters closer to the bars as well if needed. However, if an 80 stem works, that's a pretty short stem and would suggest you could defintely go with a smaller frame (assuming your other body lengths are not way out of proportion). A shorter stem will still work just as well as you probably know-certainly a much cheaper option!!