Yes, there is something wrong. That mount point lets it move. Not a good thing. That's the reason to mount it to the frame.

Never hang it on a part that can move independent of the rack itself.

Move the mount to the frame, or the seatpost, below the suspension part of the seatpost......

If you get a bolt on seatpost clamp, the size of the seatpost, you can attach each side on it's respective side of the clamp. Mount it as close to the bottom of the exposed seatpost as you can. Or, "P" clamp it to the seatstay bridge, or the seatstay itself..

I don't see any mounts on the seatstays**********? Odd....