the problem is
the rear derailleur keeps trying to spontaneously shift into the next sprocket when under moderate load- then it drops back to the original one.
(sort of like an old friction shifter might do if it were between gears.)
it never does this on the bike stand.
it never does it in first gear.
it seems to do it a bit more often when on the big chain ring.
it seems to do it a bit more often when the down stroke is on the drive side.
Sometimes when it happens it will do it once and then be fine for some time;
other times it happens repeatedly on each pedal stroke for some time.
i asked my local mechanic to look at it, he saw nothing wrong, but did reset the derailleur
back to the factory defaults on the theory that perhaps it had gotten into some strange shifting mode. but this did not help.
I have a spare wheel with its own cassette, so I tried that using that on the theory that perhaps it's a problem with the cassette, but the problem is the same on the alternative cassette and wheel as on the original.
I weigh 16 stone if that has any bearing on the issue.
the drive group set has about 3000 km on it, except the chain and rear derailleur which has about 1000.
the frame is titanium if is of any import.

It seems unlikely this has anything to do with Di2 shifting. More like a mechanical issue with the cogs or chain. The Di2 system doesn't have any way to detect load on the system, so I don't see how it could be reacting to you applying load.

I'd make an extremely close and detailed inspection of the chain. Look for anything out of the ordinary. I had a weird skipping issue early in a 50+ mile ride. Only later did I discover one plate of the chain was cracked and the skipping was caused by one side of the broken plate twisting around causing problems.

Any possibility you have the wrong speed chain on the system?

Also, Di2 or not, a bent derailleur hanger a first place to look for weird shifting behavior. Bike shop should have checked that though.

The way the problem is described it seems the chain is lifting off the sprockets when under tension. You have done the mechanical check on the cassette, but have you substituted a new chain to test?

First thing I think of is whether the rear der has had its indexing set properly. When the guide pulley (the top one that guides the chain onto the cogs) isn't directly under the intended cog the chain can snag (what is actually the very beginning of a shift, whether the rider is knowingly wanting to shift or not) an adjacent cog and or try to lift off the current cog. This possibility is increased with any flexing or other sideways movement of the system, and we all know that the bike does flex sideways with each power stroke. The comment "it seems to do it a bit more often when the down stroke is on the drive side." matches my experiences in that the strong leg will push harder and thus flex the bike more.

DI2 has a feature that centers the guide pulley under a cog to set the indexing. The der/pulley can be jiggled inward or outward as needed during the set up. If this was set correctly and the wheel/cogs are changed or removed and not reinstalled exactly as before the new relationship between the guide pulley and the cog could be slightly different. We see this when the cassette is replaced as an example and at least check this adjustment before letting the bike go.

Of course the problem could be caused from other another issue, or even a combo of different issues. bent cog teeth, deformed chain/links, worn out guide pulley bushing and of course the usual first thing many here will claim- a bent der hanger. Andy

Ask Terry at Bettershifting.com

Some bikes shops don't see much electronic shift components. Perhaps your mechanic is one of them. Here, they have a lot of Di2 bikes. And when ever I talked to the mechanic that sold me my bike, we always got into a lengthy conversation and it sounded like Shimano provided endless help to him and other mechanics there whenever they had a Di2 question. So if your mechanic isn't getting in touch with Shimano on whatever regional level is appropriate. Then perhaps it should be encouraged.

However I too sort of think it's not related directly to the Di2. Just maybe a poor adjustment. Which in their manuals for Di2, Shimano give pretty good and concise instructions for set up. Did your mechanic follow those? Saying that he set it back to the default settings doesn't sound like he has a lot of experience with it.

Also might just be the bike got knocked on the RD and the hanger twisted a little.

thanks for all the advice, the problem turned out to be a very slightly bent derailleur hanger.
i second mechanic detected and corrected the problem.