You can continue to run the chain through the derailleur, adjusted to stay in one gear. In this case, the derailleur will keep slack out of the chain. Alternatively, you may be able to shorten the chain and bypass the derailleur. If your frame has vertical dropouts, it can be tricky to get the chain tension just right, but there are ways to make it work. Check Sheldon Brown's writings on this, right here ---> Singlespeed Bicycle Conversions