what kind of jacket are you using? i have a pactimo rain jacket which is just a thin waterproof plastic jacket that easily stuffs into a jersey pocket. if yours doesn't fit in pockets maybe get something lighter weight and layer under it instead
here's mine, it also has vents under the armpits and back and lining on the collar. they only cost ~20$
