Voiceover add-on to save mouse position

By Adam Yu, 3 August, 2023

Forum
macOS and Mac Apps

Hi,
I was using some apps that are not Voiceover compatible, and I have to move the mouse pointer to click buttons. I was wondering if there's any add-ons for Voiceover that can save the position of the mouse pointer in an app. Its function is just like hot spot, but it saves the position of mouse instead of Voiceover cursor. If there's no such app available, I may be able to develop one. Does anyone have suggestion for this idea?

Thanks,
Adam

Options

Comments

By Mathieu on Saturday, August 5, 2023 - 15:32

Hi,

Maybe you can use Keyboard Maestro for this? I've never tried, and I don't know how it behaves on a different computer, with a different screen size.
PS: I’m French, sorry for my English :)

By Adam Yu on Saturday, August 5, 2023 - 15:32

Hi Mathieu,

Thanks for the info. I've check Keyboard Maestro's website. It didn't mention anything about mouse control. I guess Keyboard Maestro is somewhat like Mac's Shortcut app, and maybe it won't help in this case.

So I thought that maybe I can develop one. It behave just like screen recognition on iOS, it can call the OCR API to recognize the elements on the screen and move the mouse cursor to them. It can also save the mouse position as hot spot so that we can control the app even if the OCR doesn't work.

PS: Je parle français aussi. :)

Adam

By Cliff on Saturday, August 5, 2023 - 15:32

Wow, fantastic idea! Would be awesome if you could get something like that to actually work! I use VOCR almost daily to try to navigate inaccessible apps, and it helps a great deal in many cases. But if you could further build upon some sort of OCR framework and be able to save custom mouse positions, that would be somewhat of a revolution! And if then the saved mousepositions then could translate as accessible buttons that can be labeled, well then we are a good step closer to make very many inaccessible apps close to accessible! :)
I'Ve used Chat GPT to make a few simple scripts with great results. But my understanding is that chat gpt is really badass at more complex coding tasks too... Perhaps you could get some great coding inputs from describing what you want made to chat gpt and see what it spits out for you! :)
Anyways, I'm looking forwards to see what you can come up with! :)
Cliff