Your first instinct is probably right. In general, you have to remove links from the chain for it to be the right size - wheelbases vary in length (among other variables), so everybody needs a differently sized chain. It's expected you'll have to shorten it.
Now that you've connected it, you'll probably also have to get another connector pin when you re-break and remove links (unless it's an SRAM chain with the connector link). EDIT - nevermind, I see that KMC chain, it has a quick link, correct?
Only other possibility - are the cage lengths on the two derailers comparable? That can cause a problem if the derailer cage isn't long enough to take up the slack. But in your case, it should be the chain.
Easy way to check which it is - when you put the derailers in the big/big combo, is the rear derailer fully extended? If not, the chain's too long for sure.