There is a Fall season in Toronto? I thought there are only two seasons there: August and Winter

Seriously though, I don't think you need to over think this. Just get a wind vest (any will do), full finger gloves, and a pair of arm warmers (wool would be best), you are set. The single most useful piece of cool/cold weather cycyling gear I have is a neck gator. It works wonders, and I would suggest you take a look. With that, you can ride into the low single digits.