I agree that stuffing a jacket in one's jersey pocket isn't ideal. I did that recently with my Pearl Izumi rain/wind jacket that packs nicely into it's own pouch and I couldn't stand riding with it stuffed in my jersey pocket. It's hard to compact a long-sleeved anything into a size so small that it's not noticeable.

Related to a different post I created regarding carrying extra water bottles, I think a good solution for temporarily needing to carry more stuff (water, clothing, snacks, etc.) is the CamelBak Chase 8 or similar backpack. I know that wearing a backpack initially sounds annoying as heck, but I've found that after a few rides with a small backpack on, I barely noticed it and it drastically increased my carrying capacity w/o adding bags and accessories to my bike. In addition, I can wear it hiking and anywhere I want extra supplies.