I use an HTC Incredible Android phone with the My Tracks App. I find the accuracy to be pretty good, usually about 1/10 to 2/10 of a mile over a 25 mile ride as compared to a calibrated bike computer.

When I upload to Google Maps, you do see that the route does not track on to the on-line map accurately, in that the track is on the wrong side of the road in many instances, or slightly off-road in some cases. That is not an inaccuracy in the track, but instead an inaccuracy in the on-line map. Which makes sense when you consider that the on-line map is hand drawn - I.E. using a computer program, but still done by a human. You can see discrepancies when you toggle between satellite and map view.

Thus it's easy to see where an on-line mapping program will have an error as well, especially since all (excepting Garmin and other GPS manufacturers) seem to use Google Maps as there data base.

As well, the GPS devices themselves induce errors by losing a signal, then assuming a track, then re-acquiring, then re-drawing the track, all the while the device is dropping "breadcrumbs" as it where, as track points. The more breadcrumbs the better the resolution. My old Garmin Vista had terrible tracking resolution as compared to the Google Maps app in the phone. It seemingly is related to the amount of allotted memory for beadcrumbs, with newer devices having more memory available, thus better resolution.