roadfix

I think your idea sounds fine with use of a tandem skewer. Your frame's flush vertical dropout is perfect for this but as Revtor points out, I would be concerned about deformation of the rack legs. The setup is basically similar in principle as the OMM rack. Of course, removing the rear wheel means pulling the skewer entirely out but that's no big deal.

EDIT:....12 hours later.... On second thought, don't do it unless you first mount your rack mounts on small thick aluminum plates. These plates will provide enough secure surface contact area for the skewer to bite into.