Like you (I think), I'd rather stop wearing lycra too early than too late. I've opted for MTB pants which are very comfortable. At times I wear lycra shorts with tennis or similar shorts over them so only a few inches are visible near the knee - compression like. I still wear bike jerseys but have moved to loose fit.