Maybe most people just look at Specialized as a sort of a giant "bike stuff" conglomerate, so focus on ther bikes is a bit "blurred" with C&Vers. This is further muddled up by the fact that they contracted out the building of their bikes as others here mentioned.
I betcha if they just sold bikes and not tires, helmets, clothing, components and other accessories. People will see them more as a "bicycle maker" and their bikes will get more attention from the C&V community. Nearest thing I could compare to Specialized was "PBS" and their outsourced bikes in the 80's and 90's......Not too many look for those bikes from that company these days although I think most of them were admittedly quite good quality.....