I had this happen on my bike as a result of tightening down a wheel with broken locknuts, causing it to bend open. You're going to have trouble keeping your wheel from sliding forward in the dropout, since there's less dropout for your locknut / nut to grab ahold of.
How to close the dropout (assuming it's steel): get a medium sized c-clamp and some thin pieces of wood if you care about your finish. put the surfaces of the clamp on either side of the dropout (the deraileur hanger works well, if you have one, for the bottom contact area) with the wood between the clamp and the metal. Tighten the clamp, and measure the dropout space as needed (you don't want to overdo it). Once it's been bent into it's correct position, it should be good as new.