Thanks for all the great ideas, gents!
I'll take the derailleur off and see what you're describing and let you know. It's possible I reassembled the cage wrong.
There's no adjustment screw to allow changing the angle of the derailleur. This was a lower-end bike.