Thank you for your advice everyone. I was able to take it off with a LOT of time and patience by first hitting from the top to move the cone a few cm (this after spraying wd-40 before I went to bed). Then I woke up and hit around the top of the adapter to "loosen" it up before turning it upside down and knocking away from the bottom of the fork using an 8-inch metal bar and striking for about 15 minutes. Eventually, I felt the stem moving little by little. Once that happened, I really went to town with a couple of hard strikes then BAM! out came the adapter.

Now the real fun begins with the removal of rust in the fork.