this will come down to what you really are comfortable with. I am NOT comfortable putting my $1000 cell phone on my bike where I can break it. I am much more comfortable putting my $250 Garmin for that. If I had an crash or whatever I would rather upgrade my Garmin.
BUT if I was going to use my phone I would put it in my back pocket and ride.