I am a true cyclist, I commute daily, in most weather, I ride around 10,000 miles a year, I carry things on my bike. I am fit and committed to the sport. I have been doing this for years. As cyclist and cycle commuters we as a group, don't do this to save a few seconds, minutes or even hours. We are not in a hurry, ever, we enjoy nature pleasures as they present themselves to us, we see the world up close and at a speed that allows us not to miss much. I stop at stop signs and red lights as an example to motorists, I want them to be as patient and courtious with me as I am with them. As we all should be smart enough to know, we as cyclist could never prevail in a road rage act of aggression on their part, we will end up at the bottom of the pile. As for the clip in clip out thing, for years now I just stop and track stand, I stay clipped in and sit still in my lane, most motorist enjoy watching me and don't mind waiting, when the light turns green I just ride away. It's a skill that everyone can aquire, It's fun, I have stayed still in a traffic line for over 5 minutes and I wait my turn in stop and go traffic, all while staying tightly clipped into my campy record pro-fit pedals. As for the cardio and fitness thing, if you ride every day and everywhere you go, you will be fit and you will stay that way, on days when you unpack your bike and ride with the local blasters, you will burn their legs off, It's great.
Be patient, slow down, the world is way too fast, enjoy your sport, Be a true cyclist and live to tell your stories.