I would certainly say to find saddles that work but not all saddles will work in all situations. I ride different Brooks saddles on a number of bikes but I can’t ride any of them on a mountain bike. Even the narrowest of Brooks saddles are simply too wide to allow for an easy shift rearward on a downhill. They also tend to be a lot more slipery than other saddles which also makes them hard to use off-road. For that application I use narrower plastic saddles, usually Selle Italia Flites.

Find saddles that work but don’t be afraid to try something else.