I haven't handled an iPhone 6, but the 4s is much tougher than I'd expected. A friend gave me his old iPhone 4s and said he'd accidentally dropped it from four or five storeys up. I don't see a hint of damage. Then I dropped it from about waist height to a hard concrete or tile surface after the belt hook popped loose. The external aftermarket protective case was cracked along one corner but the phone itself was fine. The similarly sized Android phones I've handled were much less sturdy, although those have all been low to mid-priced models from Samsung and others.

Usually I tote the 4s in a padded handlebar bag while it's running Cyclemeter. The only time I handle the phone is to check a map or use the camera, while I'm stopped. I don't ride hard enough to care how I'm doing in real time, and if I'm lost enough to need to consult a map, I need to pull over anyway.