That's probably a fair assessment. Times when cadence (and therefore power) can be off are when you're accelerating really hard, or when you suddenly go from pedaling hard to coasting. In many power meters that use accelerometers for measuring cadence, you can see anomalous power readings. Another situation is when you're mountain biking at very low cadence because you're trying to "muscle up" over an obstacle. Accelerometer-based cadence seems to be confused when you're pedaling really slow at very high crank torque. But most riders really don't care much about those situations.

Not just accelerometer-based cadence can be off. I used to have a reed-switch sensor for speed and it used to go crazy occasionally, where I'd see speeds of a couple hundred km/h for 2 seconds. So I examined my data files and realized they all occurred on a certain route, a particular distance from where I'd habitually hit the lap button. So the next time I went out that way, I counted that distance from the place where I hit the button and right there in the middle of the road was an understreet power vault. That power vault was leaking EM and it was interfering with my sensor. I bet the FCC didn't like that. Maybe not the public health dept, either. You can find out some interesting things if you pay attention to when things go wacky.