Also try wrapping your aluminum brake levers with twine or a couple layers of heat shrink tubing to help reduce the amount of heat they draw from your fingers.

I also found it helped to wear one set of gloves in the morning and another for the evening commute and let both dry out overnight at home.