I have a small saddle bag worth of stuff: multitool, patch kit, tube, levers and a 2 or 3 strips of leather. My phone stays in my jersey pocket but the only thing keeping it from sliding around would be the grippy rubber case, but even that will work its way into position for launch.

My bag is pretty much stuffed, so for longer rides i put bars/keys/money/ etc in my pockets, but instead of focusing on the ride my mind is always "something just hopped up in my pocket" or something along those lines.