I don't believe in junk miles either. Every mile has a purpose whether to build endurance, suffer through while riding an interval, etc.
Beyond that you can have an easier time riding 100 miles as opposed to a 12 mile time trial. It just depends on your goals.
I have a 7 month old and an 8 year old in the house. As such I ride faster shorter distances to keep the wife happy. They keep me in shape for the occasional 120 miler with 12,000 feet of climbing. Occasional being the key word there.