I think cycling has a greater investment for entry than most other endurance/racing sports. You need a decent (not great) bike, you do need some experience, you need decent fitness (you can get away with poor fitness in a running or tri event), and you need the willingness to get up early and line up (out here 5's are usually the first race around 7AM). In reality, just to get to the starting line at a Cat 5 race, you're looking at some serious time and financial commitment. Compare that to any local 5k (about same price as a crit), where you can throw on a pair of shoes and walk the entire event, and I can see why people are more inclined to take on other events. You can't go out to a bike race and soft pedal to a finish, you'd get pulled and sent home. In a tri or running event you can come out and finish whenever you want (somewhat).
Bike racing can also be a pretty intimidating venture for some. There are a few guys that jump into our local group rides that are amazingly strong riders that have no interest in racing because they say "that stuff is way too serious" or "I'm not that crazy". Little do they know that they'd do just fine in most any race.
Without a US superstar the interest has dropped considerably. Sadly, I don't think superstars doping is really a detriment to the average viewer's interest. You see PED use in all sports and people still love those other sports. But, without a superstar to root for, especially an American one, people's interest tapers quickly (see golf and the decline of Tiger Woods).
Oh yeah, add to all of this that people HATE cyclists and as a population we are collectively getting much lazier. I see fewer and fewer kids out on bikes these days as it's easier to sit on the couch and play video games. Fewer kids riding leads to a significant drop in teenagers and young adults racing.
Ultimately, I think it comes down to, as Grumpy said, bike racing is really hard. People opt out of the hard stuff for the easier stuff.