The thrift shop suggestion sounds like the best one so far, if you can find stuff you need.

If Costco still has that merino sweater, run there today and buy two! I've bought a few over the years. Quality varies from year to year. The latest one I bought was slightly disappointing but still worth $20.

You could also take an old shetland sweater that is intended to go over your shirt and wear it as an undershirt. Sounds crazy, I know, but if you want non-cotton to be against your skin, it's one way to do that.

Also, sporting good stores are likely to have decent stuff at below-pearl-izumi prices. The difference might be fit. Cycling clothing generally is long in the back and sleeves. That shouldn't be a huge problem, though.

If you can, look for stuff with raglan style sleeves. They allow better movement in the shoulders when you're in a cycling body position. Not a big deal, but if you can find it, you'll be glad.