How loose is the fit of your tire to the rim? I can't see that being normal since you never really hear about it. As said before, check the stem-hole in the rim for sharp edges, imperfections, etc. Are you using a schraeder valve with a schrader hole? I can see some forces being mean to the stem if it's a presta in a schraeder hole, maybe? Trying a different brand of tube may also help.