I wouldn't have thought you would need to play with the B-screw like that - after all, that rear derailleur is designed to work with a cassette that size. Do you have the derailleur mounted correctly against the stop on the hanger - some Shimano MTB derailleurs have 2 bumps on the derailleur link that can be confusing - comes up all the time on the MTB forum.