I love all the shortcut actions offered with VoiceVista. The developers understood you would use it hands-free.
But when I use Seeing AI, Aira's Access AI or Be My Ai in Be My Eyes, I struggle to hold the object I want described, hold the phone and push the "quick capture" or "take picture" button. I really wish there was a shortcut for that.
Today I worked on editing old shortcuts and creating new ones. I looked at the actions offered by these three apps and nothing exists to have any of them take a picture and automatically generate a description.
With Be My Eyes you can call a volunteer, but if I just want to know what color something is, why should I call a volunteer? Or if I want to know something else simple, it's faster to use AI.
Even when I spread out the item on a flat surface, say I'm sorting t-shirts or mail, I still have to take one item at a time, spread it out, make sure nothing distracting is in the camera's field of view, position the phone above the item and then push the button. It seems like there should be a way to ask Siri to push the button so my hands can be used to hold the phone and position the objects to be described.
Even if my phone is on a lanyard around my neck, I still have to lift and position it.
So I wish these developers would realize, like the VoiceVista developers did, that it is more efficient if I can perform some actions using my voice.
By Deborah Armstrong, 15 November, 2024
Forum
iOS and iPadOS
Comments
Vision Intelligence: could help
I think Vision Intelligence: in the later IOS 18.2 could help a bit.
but, we have to have IPhone 16!
Re: Be My AI
For what it's worth, with Be My Eyes, you can use the volume buttons to take a photo of the object you want the AI to describe for you. 🙂
VoiceVista developer
Is a one man army and he is a demigod walking the Earth
One workaround for hands…
One workaround for hands free control is using voice control in conjunction with VoiceOver. I realize it's not the optimum solution, but it would work in a pinch.
Voice control is a definitely good idea
Yes, that's what I used to do to speed up and simplify the taking of images and also to improve te image quality significantly. I turned on voice control with Siri ("hey siri turn on voice control") before using Be My AI, then I said "tap take picture" on the Be My AI screen whenever I wanted to have something described. I used voice control for the other relevant controls too, e.g. when asking follow-up questions - "tap ask more" etc. This way the double tap just before the actual taking of the image didn't move the device and I prevented the blurring caused by that little motion. The result were significantly better descriptions.
I suggested introducing shortcuts to Be My Eyes support multiple times and emphasized that it was not just a request for a convenience feature, but it was a key to taking high quality images. Unfortunately though they do not seem to be much interested in this, as they released a lot of updates since then, but haven't found room for this little, but very important feature in them, although I don't think it would take so much work to implement it.
Shortcuts for using Be My Eyes
There are a number of shortcuts that you can use to work with Be My Eyes. Here is a list:
Call a Volunteer
Ask Question
Read Text
Describe Normally
Describe Quickly
Describe Fully
To use any of these shortcuts you can either hold down the side button or start by saying "Hey Siri" and then speaking one of the phrases above and appending "...with Be My Eyes".
For example,
"Hey Siri, Describe Fully with Be My Eyes"
will automatically launch the Be My Eyes app, take a photo, and speak the AI response. So you can essentially use the app hands free except, of course, that you have to hold the phone and point the camera at something.
These shortcuts can all be found in the Shortcuts app on your phone.
Hope that helps.
--Pete
How did you get those shortcuts to work?
Siri does not do this for me. However, if I go to create a new shortcut with the Shortcuts app, I am offered two choices:
One. Call volunteer.
Two.: Call an organization.
I see nothing for the Be My AI service within the Shortcuts app. So, how did you get yours to work?
Be My Eyes v6.0.1
In v6.0.1, which the App Store claims I have the latest, the only Be My Eyes shorcuts available are Call a Volunteer and Call an Oranization. These other ones sound useful but you must have some secret (beta?) version.
Be My Eyes Version
The version may be the difference. I am running the beta and it is version 6.1.0.
The beta notes indicate that this version is prepping for an update to the Meta glasses. So maybe they added some new shortcuts.
Interestingly, the Settings / Be My Eyes dialog doesn't indicate anything about shortcuts. I only found these accidentally in the Shortcuts app itself. They are difficult to find there since the list is so long and I don't think many people would naturally look in the Shortcuts app.
Anyway, hopefully you'll be seeing those shortcuts soon in an official release. Could be very handy.
Sorry for any confusion.
--Pete
Update on shortcuts for Be My Eyes
In looking into this further, I found that the shortcuts that are available with Be My Eyes are actually all listed in the Settings tab of the app. So when a new version is posted, you will be able to see all of the new shortcuts there and not have to poke around inside the Shortcuts app itself.
I'm guessing people will find these new shortcuts not only useful with the Meta glasses but also on the phone itself.
Stay tuned!
--Pete
Still not seeing those shortcuts
I'm pretty experienced creating shortcuts. And in my shortcuts app all I see are the two mentioned; call a volunteer and call an organization. And I have automatic updates turned on.
When I look in settings under siri shortcuts the only two I see are "call a volunteer" and "call American council of the blind".
I have version 6.0.1 and the app store shows no updates.
I have an iPhone SE 2020.
Is this A/B Testing?
I know Microsoft does this for their Windows Insider betas. Basically, 1 group of testers would be having some set of features, and the other group would be having entirely different set of features to test. This might help them in gathering more focused feedback in initial stages. Don't see much sense in limiting number of testers though.
In my case, Siri shortcuts for BME inclued only the call a volunteer option. Call organization etc is entirely missing. I tried long pressing and what not, no other shortcut showed up in settings section of the BME.
@BME, if you are reading this, Please creat a 1 tap shortcut to open the app, take picture, and announce description. I might be able to creat something on my own, basically, a custom shortcut which opens BME as first step, then presses volume key, but, that seems like headake to creat.
If someone expert in shortcuts wants to do it and share it with the comunity, that would be great.
Curious
Sometime ago, a user on here posted that they had made a shortcut for Be My Eyes, at least I think it was for Be My Eyes, but you had to pay for it. I am wondering if that is what the shortcuts listed above are from, or if they are from another app like perhaps Piccy Bought or something similar?
This is the link to that post I mentioned above:
https://www.applevis.com/forum/ios-ipados/updated-i-wrote-shortcut-make-getting-descriptions-images-screenshots-easy-now
Beta
To clarify, these additional Siri Shortcuts are currently only available to people on the Be My Eyes beta.
They are not yet available in the App Store version of the app.
Dave
Thank you
Thank you for the clarification. 🙂