I'm wondering what I should keep in my rack bag so that I'll be prepared for some common mishaps such as flats.

are there things that you carry with your bike at all times?
such as tire repair kit (or spare tire), screwdrivers, wrenches and allenkeys.

Also, do you take your rack bag with you when you lock up the bike?

I'd get a seat bag or a tube bag, but the rack bag has a lot more uses.