You can close it in whatever position you feel like. So long as it's closed tight, it's not coming open on its own.

If the "closed" position leaves the handle too close to one of the frame tubes, it can be hard to open again, as the frame tube interferes with getting a good grip. So I usually leave mine pointing straight back.