I am in the same situation, although I am deaf in my left ear from birth. The difference is that someone riding with ear buds with music blasting has a DISTRACTION whereas someone who is hearing impaired does not. Someone with a hearing impairment usually has a stronger sense of their limitations and tries to be more aware of their situation and will likely notice a situation without audible warning that is dangerous. I know I rely on visual clues more strongly than a few of the people I have ridden with. Scanning long before an intersection, watching the road in FRONT of a vehicle that is in front of me for dangers, watching shadows and looking for movement in my peripheral vision, etc.

Someone with average hearing having music blasted into their ears may not have such an advantage and many times does pose more of a risk. I know I've come up behind a few on the MUP that I've dinged my bell repeatedly, called out, waited for a better opportunity to pass and STILL they about jump out of their skin when I do pass - and a few times jump INTO my path, once nearly knocking me off my bike.