The 650B probably did have something to do with it, but I think it was due to a a number of other factors as well, such as the lack of downtube shifter bosses and the requirement of hard-to-find long-reach or centerpull brakes, in addition to its (arguably) unwarranted reputation as the Rivendell for people who can't afford one but want one anyway.
On an aside, I think Riv may be going in the wrong direction with their product development with their bizarre specializations, most notably the idea that you need 650B wheels if you're less than 6'0" tall. I'm 5'8" all my bikes have 700c wheels. (53cm frames) They handle just fine and don't look like they're absurdly proportioned. The days of buying a Rivendell All-Rounder and bolting whatever you had on-hand to it are over. Now you have to buy everything via the Riv Reader.
(I'm dramatically over-generalizing here.)
650B is a great option, but I miss the Rambo and Rom. Where is the Riv that follows normal, if somewhat relaxed, road geometry and uses parts that are readily available at any hole-in-the-wall bike shop?