Truthfully, unless you live directly under Niagara Falls or something, the chances of actually getting caught in a torrential downpour are pretty slim. I generally tell people "I'm fine with cold, and I'm fine with rain, but freezing cold rain and wind, I don't have the clothes for." The reason I don't have the clothes for freezing cold rain is because we get about 3 days a year like that here, and chances of me riding during those 3 days are even slimmer.
Just expand your outer gear collection, eventually you will pick up pieces here and there and before you know it, you'll be riding in blizzards.