I don't want to get all macho about this, but 50 isn't very cold and there's no reason to be uncomfortable riding on a 50 degree day. I'd typically wear a not-too-heavy base layer of wool or fleece and a wind layer, probably a microfiber shell. If that weren't enough, I'd ad an insulating layer between them.
There's no need to spend a ton of money on bike-specific clothing. Thrift shops are full of old sweaters and fleece for a few bucks, and you already HAVE winter clothes, right?
FWIW, we can't expect 50 again here until April. The hot-cold-hot thing is just part of riding. Start a little light and you'll warm up in a mile or two. This isn't physics.