Took the wheel off and found the freewheel gear unit has play in it, both up and down and side to side. I can even see the bearings inside. What I thought were holes to lube turn out to be two dot indentations/marks that are 180 degrees apart and this freewheel gear unit looks to be the type that has no tool to remove it, perhaps those two dots 180 degrees apart requires a special tool like a watch wrench to screw and unscrew the unit or unscrew that cover down if it's supposed to be serviceable and re-lubed ? I'm inclined to believe the former and that this is a sealed unit, there is no play and that when it gets this bad, you are too simply removed the old freewheel gear unit and replace it for whatever a new one costs.

No wonder it was grinding away, who knows how much more I'd need to grind it before it seizes and no longer freewheels and becomes it's own fixed gear ?

Then there's this method:

http://www.parktool.com/repair/readhowto.asp?id=46