I am not a doctor. This is not medical advice, only my experience.
Big mountain bike crash in 2002, broken clavicle in the middle third, just one fracture. It was expected to heal, but after 4 months of pain, had surgery which required a bone graft from hip due to the fact the broken ends of the clavicle had long since healed. Bone grafts come from your hip bone and hurt much worse than a broken clavicle or surgery to install the plate. Had I got the plate installed at the time of the clavicle break I would not needed the bone graft, and would had not been in pain for 5 months, only one.
If you and your Doctor (not some wahoos on a bike forum) decide against surgery, at least get a top quality sling/imobilizer like a DonJoy Ultra, that will hold you shoulder back with the clavicle in good alignment....not some 2 dollar blue cloth bag you got at urgent care.
I've got a plate, 7 drywall screws, and don't notice it unless something hits exactly where the plate on the top of the clavicle.
A Cycling friend of mine just broke his clavicle in 2 places, had surgery the same day by some amazing luck and was back on the bike (I'm sure against the Ortho's orders) within 2 weeks, just like Frank Schleck, et. al. I guess the trick is to have multiple fractures, like Lance A, in which case surgery seems to be the norm.