It is that simple, but with caveats. What brand and model fork? Air fork or coil-spring?

It's ideal if you can find and download and read the service manual for your fork. Chances are it's online.

It's worrisome that you turned a knob at the top and a bolt fell out the bottom. It would be prudent to get someone to look over the fork and maybe service it and make sure that it's bolted back up properly and that nothing else is amiss. Have you ever had the fork serviced? This might be a good time.