I second the idea of getting a second opinion.

If the docs are all telling you to take time off from cycling, then take time off from cycling. Have them tell you some other activity you can do in the meantime, so you can stay in shape.