This is among the worst complaints I've ever heard. If you're 4-6% BF without trying 1st off you're so lucky you don't even know. 2nd, if you're lifting stopping is going to make you lose weight faster than anything else (muscle weighs quite a bit more than fat). 3rd, No, you can't sit on your ass getting fat and expect to keep cardio fitness.

Basically youre only option is to keep cycling, start lifting alot more to build muscle, and start eating absolutely massive quantities of food.