It's generally pretty frustrating to charge a phone directly (hence why panels are usually used to charge a battery which then charges the phone). As the solar radiation hitting the panel varies with clouds, angle to the sun etc... there may or may not be enough power to charge the phone. So it goes in and out of charging mode and generally, doesn't get there too fast. It's just not efficient.

J.