Problems with AirPods and phone calls

First of all, I should say that I’m running the second version of the AirPods, an iPhone 11 and an Apple watch 6 running iOs 14.4 and Watch OS 7.3. I’m having problems with phone calls with my AirPods. Most of the time, I’m listening to audio, mostly audio from audible or occasionally music. When the phone starts to vibrate, I don’t necessarily hear my ring. Even if I tap the AirPod to answer the phone, I don’t hear any noise. Sometimes, this happens even if I’m not answering a call. I found that it’s almost impossible to make a phone call using the AirPods. Sometimes I succeed, sometimes I don’t. I’d actually say that I have more problem making calls out than answering calls. If the ringtone starts on my phone, I can usually answer successfully. It just depends on whether the phone actually decides to ring. Has anyone come up with a workaround for this? Also. I do have a couple braille displays connected via Bluetooth, but they’re not usually running at the time.