Something different is coming: a progressive web app for you all.

All of the above!

The fact more apps from an accessibility standpoint (me being an Android user) aren't web apps, your idea is a golden glass of fresh air here!

Making this a universally accessible application is boss! I for one cannot wait to give it a try. 😎👍

Good job!
I am ready to beta test it!

Stephen, it sounds like a good way to do it. I've been having to ask questions of AI in a grid, or to be exact, several different kinds of grids such as thirds, to understand pictures I take and edit.
An issue I have with another explore by touch AI app is that there is no indication of where the image ends at the top and bottom of the iPhone screen, and as I work with several different aspect frames with lots of blue sky that the AI does not speak, I have to guess a lot. I would think this would not be an issue in your grid system.
I find myself constantly having to ask if the edge of the image cuts part of a bird or other critter off that the AI has said is in the picture, they almost never say this up front. I also have to ask a lot if something is in focus with one of my AI describers.

This reminds me of an app named Image-Explorer, but it no longer works even though it is still available in the App Store. I will be looking forward to this cross-platform app and it'd be interesting if it also worked on touchscreen laptops.
@OldBear, what's this other explore-by-touch AI app you mentioned, if I may ask?

You can't add the option to provide audio feedback as the user moves the finger around the screen, right? You know, the type and timbre, volume and other characteristics may indicate certain visual properties. I also thought of 3-D audio or spoken and perhaps even haptic feedback but those might be more challenging to implement. I acknowledge that audio feedback requires the app to treat the image as a whole as the user should hear continuous beeps or loops or blips or whatever as (s)he moves the finger across the screen and as colors shift and the level of brightness fluctuates etc. so you might need to develop a new underlying approach to redesign the interface for this to work properly.

I was talking about the Seeing AI app, in Descriptions>Brows Photos, or something like that. There's an Explore option that gives haptic and audio feedback on some of the larger objects in a photo if the process recognizes them.

Will this have Braille support, for those persons whom are both deaf and blind?

So is it possible to add the option to zoom in or out and let the user explore by smaller or larger units/distances, even pixels? You know, this will be quite handy if you somehow implement audio cues/beeps. So what I'm talking about is something like a combination of different methods usable simultaneously. Let's say you're exploring a photo featuring a person. You'd get more detailed audio feedback as you slide your finger, but only when your finger moved over a different body part or clothing would you get spoken feedback. So this will require that the app detect individual objects and describe them only while your finger is on them, by taking into account the size and location of each object, rather than dividing each and every image into the same number of zones and treating every image as a grid. One object may span multiple zones on that artificial grid, or it might be so small that it fits within one zone, so that system unfortunately didn't sound so realistic and effective to me. The alternate method I'm proposing is more like that found in Image-Explorer in that respect. The audio cues should also be heard more naturally and continuously, so representing an image as a grid may prevent that. Let me try a different explanation to clarify my point further: Exploring an image represented as a grid sounds like navigating a table with a certain number of columns and rows. So it's more of jumping from one cell to an adjacent one as you slide your finger than exploring the entire image as a whole.

Well, apparently this is where your app will excel. Whenever we have a suggestion, bug report etc., we just fire away and you take care of everything without ever dealing with app store policies, having to submit your updates and wait for them to be approved.

Now that I've left you to deal with my volley of suggestions, I'm now beginning to think of how many different scenarios in which this app would be highly useful, from exploring the world map to taking or finding a photo of a street to get a better overview for easier navigation or examining a photo taken by a friend and posted on social media in detail.

That's great.
What Enes Deniz describes, and I guess is now implemented, is much like what the Seeing AI Explore feature is, though that is very limited.
I use it for when, for example, I am cropping a picture of a bird with its wings spread to make a photo printout on a specific size paper, and I need to be sure the bird is large enough and in the desired spot without it being cropped by the edges. I locate the bird, say it's in portrate orientation, and run my finger across the screen over and over until I have a good idea of how it is relative to the sides of the picture. Doesn't work with top and bottom as well, but I can at least tell if it is in the top half or bottom half.
Having more specific details included in that would be a game changer.

this sounds amazing but remember we need our imaginations to "see" an arm, "see" a plant, "see" what we are doing say, counting teeth, it will require people to have exceptional spacial awareness, imagine a day which will never come in my lifetime where you can "physiclly" somehow? feel a photo. remember we are examining the screen which is fabulous, but it will require people i guess, to imagine they are "in" the photo, does that make sense?
so, if say you have a picture of a dog, in a living-room you'd need I guess to imagine you are in that living-room physically to "feel" how the photo looks. I look forward to trialing this, will it be this year I wonder?
I would have liked to explore my mums house decorations, in particular her Christmas tree it huge apparently lol. it will be great to get a "feel" for my childrens faces as well, yes I can touch them but it will be great to get a feel for it.

I love the idea of the food as well, say if we are in a restaurant if it can say your steak is at 2 o'clock your fries/chips in the UK, chips I mean are say 6 o'clock, for those who value that, it will be fabulous.

I have to say, I find this app that you’re talking about to be quite good. Could this thing help me identify menus on my Casio CTS 1000 V keyboard? I have trouble with the menus because there’s no speech or clicks or beeps or anything. I had to use ally to help me pair the Bluetooth connection so I could use the speakers as a sort of audio speaker to stream stuff from my phone to it. It would also be cool if he could read me what styles are on the display because this has no numbers either, it’s buttons a dial and more buttons. I have an idea as to what some of the buttons do. But the menus and learning what styles are what is a little bit difficult except for the pop styles and the rock styles

I just tried the app, and it works really well especially as a first beta release! It describes the surroundings better than Gemini or ChatGPT. Thanks, Stephen, for your efforts. My observations so far on an iPhone 16 Pro Max:
1. Maybe I'm lost in the Settings window, but it seems that I can't alter the voice. It uses Samantha, and I want to use, say, Alex, Eloquence, or eSpeak-NG. I can't find a setting for that. The only thing I see about the voice is changing the speed.
2. The AI Assistance feature cannot be started unless the Camera mode is activated first. I don't know if this is by design, or a bug. I think the AI assistance should be started independently to avoid confusion.
3. If I enable Continuous scanning from the Settings window, which scans the environment every 4 seconds by default, the app doesn't provide responses in a meaningful or useful way. After enabling that from Settings and enabling the AI assistance feature from the main window which requires enabling the camera mode first, the app keeps beeping continuously. As I ask it something, it tries to provide an answer, but the answer gets cut off immediately. The same happens after the next question. So something weird should be happening there.

Stephen, people on Mastodon who don't use AppleVis want to know if they can contact you via email for feedback and issues. Is such an option available?

I've tried the beta and It gets things write which is nice, but I turned on the live ai mode, the seccond live button, and it mentioned voice commands, I can't seam to finds them? I ask where are the whipes, get nothing back, what text is on the whipes, same thing,, also the room description things my room is empty when it isn't.

Both the idea and the current implementation in my short initial test. I can totally see the potential and this can go places if put together properly and cleenly. I'd really like to see large-to-small maps being able to be described this way. Also, as you said, this community can be tough to impress so I'd be prepared to face the toughness as an inicial alpha is opened up.

Thanks, Stephen.
1. As for speech, any chance of using other voices already available on the phone? It can be Alex, for instance, for iPhones. I mean phones provide access to a number of built-in voices not just one.
2. As for communication, I'm perfectly fine with AppleVis. However, guess people on Mastodon prefer an email address if doable for you.
3. And thanks for looking into the constant beep/ cutting off issue.

I think a good many blind people use Discord, so a Discord community could be a good discussion group for this.

First, the web app works pretty well. I tried the room view and a photo. They both worked well, and I was surprised at how having a kind of tactile view of the photo helped me remember the photo even better. Fall leaves at the top left and right, text of the flyer in the middle. Stuff like that. I could imagine using this in video games where there's a game board. Of course, web apps can't really work on top of games but I could take a screen shot. I imagine the photo picker would let me get to all my screen shots.
Now an idea: I didn't know when I'd gotten to the last element of a photo, so maybe a boundary noise? when there are no more elements below where the finger is?

First, I love the detailed description of the tool. It's clear a lot of thought was put into it. I never thought of this approach for describing images, but after the different scenarios you described, it makes sense, and makes me wonder why this hasn't been tried already.

Since this is a web app, I'm also wondering about a future possibility of using this on a desktop or laptop computer, and instead of capturing the camera, it would capture the screen. A user could use the mouse or touch screen in the same way as an IOS screen. This could be very helpful for exploring graphical detail in more detail. It could also be really helpful for exploring inaccessible content, like those stupid anti-virus install wizards that are rarely accessible, when I want to remove them. Once the user has identified a checkbox, button, etc., the user could then tap, double tap, or click/double click to activate an inaccessible control.

I'm looking forward to giving this a try, beta, or at release.

Can language support be expanded beyond English? And why is Apple not among the options to sign in quickly? This is honestly surprising and somewhat strange as you're posting about this app on AppleVis, an Apple-focused forum, yet Google, Facebook, Microsoft and e-mail sign-in are all supported, while Apple is not.

I don't tink I'd use the web app a lot myself but, I could see this replacing BeMyEye's AI feature on windows.

Imagine you take a picture and if you want to hear the text,, you can, or if you want to explore it, you can with arrows and directional audio, that would be fun.

I'm thinking like reddit posts, so far a lot of the describer Addons for NVDA describe to much in my oppinion, if i go on r/shitamericanssay for example and get a screen shot described; i odn't want to know, it was written on tuesday, and hte post has 100 upvotes, I just want to get to the post.

I'll admit I won't use it much but it'd be nice as an NVDA adon for when i need it.

Here's what I got on my Windows computer: "Location unavailable. Make sure Location Services are enabled in iPhone Settings → Privacy → Location Services.". Since this is a web app and this is one of the features you highlight to promote it, in-app messages/responses should not provide specific references to one platform only.
PS: Here's another such message: "Important: Please turn off VoiceOver (iOS) or TalkBack (Android) before using exploration features. The app provides its own voice guidance.".

So I've noticed a small issue, when I go to explore any of my photos. Once I have loaded up a photo, and disabled VoiceOver on my iPhone, I will use the built-in speech engine to explore the image. When I'm finished, and back out, I go to turn VoiceOver back on, and I get a dialogue asking me to grant camera access. Note that I have already given the camera full access for this particular web application.

Is this a bug with the AI assistant, or an iOS/VoiceOver issue?