First of all... don't worry about looking like a dork. The most important thing about what you are wearing while biking is... is it comfortable? Is it helping (or at least not hindering) my enjoyment of the ride?

I recently went through the purchase of a new bike after years of not having one and I made the oh so common newbie mistake of purchasing a hybrid. Then I took it back and got another hybrid... then took that one back and settled on a cyclocross bike.

Reading what you wrote about what you want to do, I'd highly recommend considering/asking the bike shop to let you return/exchange/whatever and trying a cyclocross bike. They are built for speed, yet can handle rough terrain. I have brought mine on group rides (not competitive racing rides but group rides where everyone else has a road bike) and I was able to keep up with relative ease. I have also taken that same bike (and tires!) on crushed limestone paths (varying in their looseness) and my bike did great. (very loose gravel was a little nerve wracking but I made it through) My boyfriend has a full on mountain bike and I was able to keep up with him. (I think he is a little jealous)

Anyways, my point is that even in your post you mentioned that the majority of the time would be spent on roads and biking for speed/fitness. I think you'd enjoy a road-ish bike more.

Does the bike store you ordered from have any policies regarding trial period and such? Most bike stores do. Mine was a full refund within 30 days ... this allows for people to actually ride the bike and try it out (impossible to do on a 10 minute "test" in the store parking lot).