I use a beany/scull cap under the helmet as the temps get cooler, mostly to cover the ears. For winter cold near the freezing mark, I use a windbloc Peruvian style head cover good for most any temp. I also in very cold temps use a face cover and there are many types of these such as the Buff or balaclavas.
Base layers come in various weights with the warmest having a fleeced inside layer. Before it gets really cold, often a lighter base layer with arm warmers underneath is enough. As temps get colder, a wind block top or vest can be added as well as tights.

For gloves, I mostly use light liner gloves with a fleeced inside and carry a pair of wind proof gloves and as temps drop even more, I use ski gloves. On my feet, I use hiking type wool socks with insulated foot beds. https://www.grainger.com/product/34GR23?gclid=EAIaIQobChMIu-zZ3OyD5QIVDbbICh2uvQx-EAQYASABEgKWufD_BwE&cm_mmc=PPC:+Google+PLA&ef_id=EAIaIQobChMIu-zZ3OyD5QIVDbbICh2uvQx-EAQYASABEgKWufD_BwE:G:s&s_kwcid=AL!2966!3!281733071171!!!g!537545367267! For the coldest temps I add booties with a fleeced inside over bike shoes.

All this gear can be added to or stowed in a trunk bag riding on a rear rack. I have a handlebar bag that can fit most of this gear but the trunk bag can fit a gallon of milk when shopping so that is what I mostly use. Others prefer to use a back pack in the appropriate size. You will not be surprised that some expermenting will be needed.