I am using an iPhone SE 2020 with the latest IOS version 14.4.1
I get many image forwards on my What app. application. As of today, I have
image recognition turned on in my VoiceOver Rotar settings. However, each
time I want to get the text out of the image, I have to open it, switch the
Rotar to Screen Recognition and turn it on to be able to read the image. Most of the time, the text read out of the image isn't accurate and unreliable.
I also use another tool by Microsoft called Seeing AI to read text out of
images but this again for which I must first open the image file, click on
the share option and then again choose the share option and swipe till I get
to the Recognize with Seeing AI item listed in the Share Sheet.
All this does enable to read text out of the images but is a time consuming
and effort to do. I was wondering if there is a way to automate this
behaviour by which I mean on opening the image it automatically starts the
recognition process using Seeing AI or Image Recognition of the VoiceOver
accessibility. OR if there is a way to put the Recognise with Seeing AI
option in my Favorites in the share sheet; perhaps a way to customize the
Share sheet options.
On the other hand, I have been exploring 2 more resources on my IOS device
that runs 14.4.1 which are:
1. Shortcuts
2. Activity Settings under accessibility VoiceOver
In the shortcuts application I could find an option that says when opening
an app. do this. That option does not allow me to choose a file type but an
application which will trigger another action. This would not work to my
requirement. I tried exploring the variables for image functions but did not
come across any that suit this objective.
The activity settings help change VoiceOver behaviour when a application is
launched; here too I could not find a case where if an image is opened, do
the recognition or automatically launch the Seeing AI app.
So the question still remains open, is it possible to automate such a
process. Ideally I should be able to open the image file from the what app.
message and automatically Screen Recognition should extract the text for me
to read or launch the Seeing AI app. to extract the text from the image.
Would appreciate any ideas on the above requirement and idea. Have a good
one!
P.S. Have already mailed this to the apple accessibility team; will comment here if any ideas or suggestions are provided meanwhile if anyone knows how to achieve this, it would be really awesome to know
Comments
There really isn't any way…
There really isn't any way to automatethis at the moment I'm afraid.
You can edit the share sheet, which I've done. go all the way down to the bottom andselect edit. Then you should be able to move the seeing AI option up the list a bit, which should at least make it easier to find. I've found making sure the share sheet actions you really need are near the top and learning where they are on the screen so you can tap them without swiping through the whole screen helps a bit, too.
Thank you
I thought as much but was hoping if someone could probably figure out a way of doing it using the shortcuts app. This is why I have mailed this to apple accessibility, this is not only an accessibility feature but a function request. You can't customise too much of the share sheet. I have noticed apps. I don't use also show up on the Share Sheet by which I mean that the app. is installed but I don't have it showing up when I edit the Share Sheet to remove it. I am guessing it has something to do with Siri suggestions when sharing. Would appreciate if you could share your comments on the idea, is it worth it requesting such a feature or it's silly...
this app may work
all I can say about this app is that it is accessible. I haven't figured out exactly how it works.
Data Jar
Data Jar is a data store designed to be used with Shortcuts. The data store lets you persist key-value pairs. If you have used dictionaries and lists in Shortcuts, then you’ll find that the data format used by Data Jar is very similar. - Extensive support for Shortcuts, enabling you to read and update data directly form Shortcuts. - Store text, numbers, booleans, lists, dictionaries and even files. - Files are stored in iCloud Drive and can be accessed from the Files app. - Take backups of your data from the in-app settings. - Support for multiple windows and drag and drop on the iPad. - Synchronisation between devices using iCloud. Data Jar is free to use but contains a tip jar where users can pay a few bucks to support the developer.
https://apps.apple.com/us/app/data-jar/id1453273600
The thing is, you can create…
The thing is, you can create Shortcuts that trigger from places other than the share sheet, shortcut keys or voiceover gestures etc, but they don't have any context. In other words, you come across an image in Twitter or WhatsApp or something and trigger a Shortcut with a VO gesture but that Shortcut doesn't have the image you want to work with. Something like this would require quite a lot of rethinking about how iOS actually works, I think.
As for the share sheet, remember there are two places you can edit it, the first section where it's mostly just open with xxxx and the second section where all the individual Shortcuts and things like process with Seeing AI are.
It's also a bit complicated because what appears on the share sheet also relies on what you're sharing. In other words, if it's an image you'll get different things than if it's an audio file.
Finally, if you've got a lot of shortcuts cluttering up the share sheet, make sure you go into each shortcut and make sure it's only accepting types it can actually handle. In other words, an image to text Shortcut should probably only accept images so it doesn't turn up when you're sharing text and such.
I'm honestly not sure how much any of this helps.
Data Jar
I'm not sure this app would be helpful in this situation. If you write a lot of shortcuts, it's good to let you share data that multiple shortcuts might need, or to make it easier to edit data when there's a lot of it.
Heard Back from apple
Got a complicated response from apple and the feature I am requesting according to them needs to be shared with what app. I don't know where to go from here and since I did not have the time to deep dive into the mail I received I will restrict my comments to only so much. Right now being able to automate such is not possible; if it was I could choose an event trigger based on the file type that is interacted with.
I am really interested to know more on this comment create Shortcuts that trigger from places other than the share sheet, will have to understand and get more information to accomplish this.
Thank you all for the comments; appreciate it!