Doesn't sound to me like the OP is being a gram wennie, but an oz and 1/2 lb weenie. Grams add up kinda slow, 1/2 lbs add up fast. Bivy sacks and no rain gear are in my opinion are stupid light. Don't do anything dangerous. Like bring a sleeping system for average temp for the area, when you'll likely hit 20* lower than that. Same for clothes.
But bringing 4 set of cycling clothes and 4 set of street clothes is crazy heavy.
Sounds like the OP has the right idea, Pack what you think you need, when you realize you don't need it this week, send it home.
Travel light, travel fast, travel comfortable!!!