I can only use the Siri voice with voiceover on my Apple Watch

Hi all, I'm having an issue using voices with voiceover on my Apple watch and wanted to see if anyone else has ran into this problem. Ever since watch os 9 when the voices section was added to apple watch voiceover settings, I haven't been able to use any voices accept for Siri voice 3, which is the default Siri voice. Going into settings and selecting Alex or Samantha for example, does nothing. Interestingly, if I go to Siri settings on my phone and switch from the default voice 3, the issue resolves immediately. Tldr, if my phone is set to the default siri voice, none of the extra voiceover voices on my watch work. Was wondering if anyone else has had this issue as every watch doesn't seem to be affected. Any input would be appreciated.
Thanks much!