I agree with the above posts re zippered pockets and caution when washing anything with velcro.
A zippered pocket is the best solution but some phones are too big. 7Mesh has some excellent products with large, zippered pockets.
Velcro tends to pill the fabric and ruin other garments in the wash. At a minimum, I'd wash anything with velcro in mesh wash bags designed for delicate items.
-Tim-