Smaug1 I bet the lower speed was more due to wind than your IGH. When I look at the speed data from my commutes, it seems dominated by wind speed and direction.
I take a backpack every day, and I've been putting it in the basket which is on my rear rack. Every so often, I'll decide instead to wear the backpack because I don't want the trouble of lashing it on with a bungee. It's not that it takes much time, just more mental energy. Now I notice that riding is more comfortable with the backpack on my back. It's because the weight of the bag tries to twist the bike over to one side. Also I guess the unsprung weight is reduced when there's no bag on the bike. So I think my new routine is just to keep my U lock in the basket. I don't lash it in. It rattles loudly but it hasn't shown any tendency to bounce out of the basket.
The basket is somewhat shallow so you can see why I lash the backpack on but not the U lock.