I doubt that pannier weight matters little. All racks have weight ratings and I own quite a variety of racks - there is a clear difference between the ones rated for really heavy loads and more typical touring racks.

For example BF quotes a 35lbs limit on their folding rear rack. They don't list a limit on their front rack, but clearly it has one.

Depending on how you end up loading the bike with panniers, trailer and rider you could exceed a load limit on a specific part even if you are below the overall weight limit.

I'd have this talk with BF. They know their bikes and they have feedback from other riders to help gauge how your intended setup will work.