Solved it today (I think).
The problem was that the body of the mech was angled back. Shortening the chain just pulled the cage forward.
After looking at it again today I decided to take the whole mech off the frame. However, when trying to undo the allen key it instead rotated the mech. It was obvious that the body of the mech was corroded on to the bolt. I did manage to get it undone although it took a lot of force. I then dismantled the mech pivot/bolt. To get the botl out I had to use a hammer. There is a steel sleeve running through the mech body and this had rusted. If it had got much worse I wouldn't have been able to undo it and probably would have had to take the bike to a shop and have them cut the derallieur off.
Having cleaned the rust out and lubed it it seems to be working ok now. Just one of the wonderful things you can discover on a 2nd hand bike.