Wow, this what, the 3rd time this week for "I wanna tour super-fast!"
I'm not entirely clear on what your plan is though. Are you planning to travel, say, 50 miles a day? Or do several 100-mile days back to back? And how long are you going for?
Even with this absence of information, I'd say the "bare minimum" is something like:
• 2 pr
bike shorts / bibs
• 2 jerseys
• 1 pr
• 1-2 pr
• light and compact jacket
• bike gloves
• repair kit
• spare tube & tire
• patch kit
• cell phone, preferably smartphone that has maps and/or GPS
• credit card, cash, emergency contact info (preferably RoadID or similar)
• water bottles
• on-bike food
• front and rear lights
• small lock
• paper map and small compass
• 1st aid kit
That should cover you for a short tour. I'd guess you might be able to fit it in a big waterproof stuff sack and put it on a rack, and perhaps a handlebar bag. A Bento Bag can also be handy for fast access to snacks.
As Raybo points out, bike shorts don't usually dry fast, so you'll dry them on the bike while riding the next day. Keep in mind that bacteria is a major contributor to saddle sores, so it's critical to keep your shorts clean.
I'd also consider raising your stem, or swapping it out for a stem that has a ton of rise. There will be a bit of an aero penalty, but after mile 70 or so you'll probably be glad to have a more upright position. Same for using wider tires.
However, one thing to keep in mind is that an extra 5-10 lbs isn't going to kill you. So if you have room for it, there's no problem with carrying a few creature comforts like a book or regular sandals.