I have that with mine also , I have not used the wedge and have been doing so for over a year now, but I would keep an eye on the screw in bolt so as it does not strip
out. Might try to find some sortof thinner metal and drill a hole in it to line up with the hitch pin and insert all together , that should take out any excess wobble / sway