Take off most of the bags, other stuff that doesn't require tools and time to remove and replace. leave the fenders, especially if you are going to be riding with others with the possibility of rain. actual rack and fender removal Is totally unnecessary and a waste of time unless its am actual race, or you absolutely must be as fast as possible.
Only bag i had for my first double metric was my mid size saddle bag, with some flat repair stuff and a couple cliff bars. I did freeze my a$$ off for the first 4 hours though, not wearing "excess" clothing for the morning because I had nowhere to put it.