OK, here's the closest picture I can find to help show folks what I'm concerned about. It's not the double layer of fabric that bothers me -- I wear long johns under jeans, for example, in cold weather. But I find the squeeze that I get from the bottom band on bike shorts to be a necessary evil at best. It's uncomfortable, but it's the only way I can keep shorts from riding up.
So wearing shorts and knee warmers, or shorts and leg warmers, sounds really uncomfortable. The double layer of lycra, and the two gripper bands, is something I'd like to avoid.
(See diagram; the red arrows point to the grippers.)
If the same sort of thing could be done with a zipper -- the way it is for trekking pants mentioned by RoadTire -- it might bother me less. Or some other way that avoids the double squeeze. I agree, yoga pants over shorts aren't a perfect idea either, but I think I'm less worried about light cotton (for example) over riding shorts and the extra sweat it might cause than I am about the discomfort caused by doubled grippers, and the double layer of compressive fabric in between them.