I did what your thinking about doing and it worked out great. I wasn't about to put out $300 for a fit, so I kept playing with it myself. I went a half mm at a time and ended up with 2mm. It sounds like 3mm is a lot, but if that what it takes, so be it.
I've had chaffing and knee problems for years until I decided to try the shims. I wish I did it sooner. I really don't know how many saddles and bibs I've gone through, to find this out. I hope it works out OK for you. Good luck.
__________________
George