54 degrees is perfect weather! Even for my commute, which is 3 miles one-way, I get too warm when it's over 65, and I have to take off my sportcoat around mile 1.8, or go into work with giant beads of sweat dripping down my brow.

I do need to put on light glove liners below 50 degrees, and when it's under 40, I'll start to layer seriously.

Ears are particular, though. Some people don't mind cold ears; others find them objectionable. If it's only your ears, check out the 180s that xtrajack mentioned, or find a sweatband that covers your ears. You might find that everything else is tolerable if your ears are toasty.