I think it's because Specialized is a company that didn't manufacture their products, but designed and then out-sourced production.
They started with tires and accessories. Jumped onto the MTB tidal wave...very smart and timely. But approached road bikes from the high end. The Allez therefore was very limited in numbers. A great design, but I wouldn't call it 'flashy' compared to the stiff competition from Europe. Stiff domestic competition from upstart Cannondale and rising Trek, and slipping Schwinn in the mid to high end.
There were less expensive Allez-Sport models later, and the Sirrus in the late 80s. By that time, the market was flooded with mid-level models that the buying public decided on.
They managed to feature their Allez in a movie, but other than that, they didn't make much marketing noise. I think their focus was MTBs, since that was where the money was.