I have arm warmers for when I start cooler, but expect temps to rise past the upper 60s (sun), or mid 70s (cloudy). I don't know about my torso, but I definitely can't stand having arm warmers on when it gets warmer. That said, they're fantastic for when I start in SF and ride into Marin - it can be 60F in SF, 80F in Marin, and feel like 45F when crossing the Golden Gate.

As for a base layer, I've only stuck with one in cooler weather - though it's surprisingly comfortable to have a snug, long sleeve layer on when the temps are ping-ponging around 70F and you're cruising along at 30-35kph.