Gloves. Gotta find gloves that fit and you are comfortable in. It took me a few pairs to find the right ones.

Also, if you're really riding in 32-65, that means 40s and 50s...neck is a big thing for me once it gets below 55ish.

And...hydrate even if you're not sweating. Colder temps tend to make people feel more comfortable. Don't get caught off guard with regards to fluid intake.