Cache it.
I often ride from cold fog to hot sun. So, as it warms up, I take off my arm warmers (actually old wool socks with the tips cut off), and hide them somewhere. Then next time I come by with the car, I pick them up.
A good place is the "attic" of a sign like this:
but a hedge works fine in a pinch.
In the same vein, I cache u-locks and other locks at different places, so that I don't need to carry them with me (I get cheap ones at garage sales).