You can also switch to a quick release setup, and cut off the threaded part off the axle the qr runs though. I did this for a while on a fg with vertical dropouts and had no problems. Quoted from Sheldon Brown

"I used a more drastic solution: on my Bianchi Osprey. I cut the rear axle short so that it didsn't protrude past the surfaces of the locknuts. Thus, only the quick-release skewer went through the dropouts. Since the skewer is quite a bit thinner than the actual axle, this gives me considerably more adjustment room.

If the skewer is properly tightened, the axle is held in place by the friction of the locknuts being pressed against the inside of the dropouts. If this were not the case, horizontal dropouts would not be usable, since the forward pull on the chain creates a larger force against the axle than supporting the rider's weight does."

Another cause could be the pressure on your bearings could be too tight.