I'm not trying to discourage, and I'm certainly not trying to start a base miles vs. no base argument - those have been hashed out ad nauseum. The way the coach I work with believes, building fitness is like any building, you've got to start with a big stable foundation (zone 2/3 saddle time), construct the walls (zone 4/LT or FTP development), then top it off with a good roof (short maximal intervals like the <5' with little or no recovery types). If anything is missing then the building isnt stable. Just a WAG on my part, but maybe you're putting on the "roof" on without finishing the "walls".