Voicemail and VoiceOver

Hi everyone. I've noticed when on the phone one voicemail and are prompted to press 2 to listen to messages or 3 to delete VoiceOver remains silent and you can't move around the screen to find the number you want etc. the only way I can do it is to move the phone until its on loud speaker, then it seems to work. Is there a workkkaround without having to go to loud speaker first?