There's a good chance that rust penetrates further under the paint than what can be seen with the eye. I think I would be carefully picking away at the edges removing tiny bits of paint to see how far it goes.

In my opinion you haven't gone far enough with the rust removal. You need to get it back to the point where the rust colour is all gone.