Workaround for VoiceOver cursor getting stuck or looping over a set of elements in Safari

By João Santos, 4 March, 2024

Forum
macOS and Mac Apps

This problem has been annoying me ever since I went blind and started using VoiceOver, and is particularly prevalent when browsing math content on the web. As a result on my work on Vosh I finally realized why it happens, and am posting to suggest a workaround.

The problem is caused by some elements in Safari not being directly keyboard focusable, instead presenting an attribute telling accessibility agents which ancestor to focus instead. In some cases it seems like the focusable ancestor is very high up the hierarchy, but since they have no relevance, screen-readers will attempt to focus their first interesting descendant in depth first order, and if one of their siblings also refers an irrelevant element as a focusable ancestor, VoiceOver will repeat the same steps thus getting stuck in a loop over the same descendants of the focusable ancestor.

Since this particular issue is caused by VoiceOver trying to automatically move the keyboard focus in tandem with its cursor, disabling cursor tracking by pressing VO+Shift+Fn+3 solves the problem. In Vosh I implemented restrictions to prevent it from trying to focus elements that usually cannot accept keyboard focus, virtually eliminating the problem without sacrificing the ability to move the keyboard focus alongside the accessibility cursor.

Options

Comments

By TheBllindGuy07 on Monday, March 4, 2024 - 19:42

Have you told apple accessibility team about this? With that level of technical details ingeneers will have an easier time fixing this than other feedbacks?
It's one of my best future favorite scenario that with the development of vosh you can actually find bug and problematic behaviour in the accessibility tree / api system of apple, so like that girl with asahi linux who told apple security about a problem with their gpu driver after reverse ingeneering them and got credited for that we can get these bug fixed?
Thanks a lot again!

By João Santos on Monday, March 4, 2024 - 19:42

No, I haven't reported anything to Apple as I have given up reporting bugs to them. I've reported many bugs to Apple in the past, both through their accessibility E-mail address and through Feedback Assistant. My bugs are always reported with technical detail including code that can be used to reproduce them 100% of the time, and in most situations they just leave my report open for years and in the end just thank me, close the report, and leave the bug without a fix.

One bug in particular that annoys me as a blind programmer is that every English American voice except for Alex do spell out any word longer than 30 characters, like UIDocumentBrowserViewController. I reported this bug through the accessibility E-mail back in 2019, as I noticed it for the first time in Catalina 10.15.1, they told me that they couldn't reproduce it even with an audio recording and a script that reproduced it 100% of the time, and as of today the bug still lingers. I also know that this is not a case of software corruption because this M1 MacBook Air was my third Mac since then, I have reinstalled MacOS more than once on all my Macs, and have experienced the bug in all installations. In Vosh I intend to work around this bug by prefixing every uppercase letter except the first with a space before sending the text to the speech synthesizer, to ensure that UIDocumentBrowserViewController is correctly read as U I Document Browser View Controller.

By TheBllindGuy07 on Monday, March 4, 2024 - 19:42

You can try on this very thread page of applevis, navigate in safari only with the arrow keys in the text of a message. When you hear end, article, hit vo command h or rotor heading down, VO will loop back at the begining to the title of the message you were currently reading. Or at least this is another bug present on the latest mac beta that has always been there since ventura.

By João Santos on Monday, March 4, 2024 - 19:42

Math content, like in the first answer to this StackOverflow question about the line plane intersection algorithm causes this problem very often, but regular content, like the old reddit layout is also known to cause problems, as pressing VO+Command+J to jump from the Jump to Content link to the search textfield and then moving back using VO+Left makes VoiceOver move from the Logout to the Wiki links, skipping the links to the chat messages, comment replies, and your own profile when you're logged in, whereas moving forward from the Wiki link with VO+Right allows you to find the aforementioned links between Wiki and Logout. In both examples, disabling cursor tracking makes the problem go away.

By Ekaj on Monday, March 4, 2024 - 19:42

This happens for me particularly when navigating pages containing embedded videos. Pretty sure we're talking about the same thing here, but somebody please correct me if I'm wrong. I'll have to try the workaround described here later, as I'm rather busy at the moment. I think I'll be able to try this on the YouTube page of a video I have to watch for an online psychology course which I'm taking. I should add that this problem doesn't seem to be unique to VoiceOver, although I haven't used other screen readers for ages now.

By TheBllindGuy07 on Monday, March 4, 2024 - 19:42

So for college I have a math etextbook while my local readaptation centre are adapting things in braille.
I already had a very very positive experience with cengage platform in the past, their website is not the best, but honestly it works, period. Especially for math graph they actually have very useful description, and the actual math content is with mathml so towards unexpected accessibility level for such a public website. On windows with nvda and mathcat for braille output and access8math for speech output both together.
On Macos though? This is just buggy. Math content doesn't play nice with voiceover, we have to disable the command affecting website in VO settings as left and right arrow in safari would just change the page. But the point is that we have 0 math interaction capability for mac. Why the hell am a able to access math content easily on my ipod touch 7 / iphone than my mac which cost me $3000!
We tend to say android is not good because of the talkback fragmentation, but apple is so so much worst then I could ever have imagined.
In 2024 now macos prevent me to use any other browser than safari at 100% capacity because the support for chrome is not very good yet.
In 2024 voiceover support for webapps is disgusting.
In 2024 in voiceover (both ios and mac) we are still unable to change the braille cursor blinking speed or shape with our braille displaies.
Even Orca in like 2021 added chromium support on linux. Never tried it though but this is not normal that orca is better than voiceover.