I have certainly notice a huge difference on my Fargo when I am in road touring mode vs. light camping/bikepacking mode. My average speed is usually 11-13mph when using racks and panniers, while with the same tires and all but running a basket and saddlebag setup I rolled at 14-16mph average.

I think a lot of the reason is that the width of your load is minimized on a bikepacking rig. You are still cutting thru the wind rather than riding like a sail.

I will say that packing the needed equipment in a wise way is much more difficult for me in a bike packing setup.