Introducing Be My Eyes' Virtual Volunteer: A Demonstration of Some Game Changing Real-World Use Cases

While it still doesn't work, the message that gets displayed when you have a photo of a person has changed. Now, it says that this photo has been blocked because a face or a person was detected in the photo. I suppose this is much more accurate than the previous message.

So, one of my friends just learned how to share photos through the sharesheet and has access to all facial information and nothing is getting blocked. She lives in Canada, but I know the app does not have location access. Is it possible that it's able to do it through wifi? We've tried it with multiple photos and it seems to work just fine for her.
Edit, some photos have gotten blocked, but others not.
Here is one which was described a few minutes ago.
The picture shows three people taking a selfie at night. On the left, there is a woman with glasses and red hair, smiling. In the middle, there is a man with short dark hair, wearing a black and grey checkered shirt, making a funny face. On the right, there is a woman with long dark hair, wearing a leopard print top, looking at the man with a playful expression. They seem to be having a good time. The background is dark with some trees and a wall, indicating they might be outdoors.

I hope this means that there will be fixes soon for this problem. I have tried to describe some photos just now but still the picture is being blocked.

As I said, it doesn't seem to be consistent but hopefully this is a good sign for us.

Yeah, I already mentioned that the other day, I just turned off my Wi-Fi, using my cellular data, and then I went to the Be My Eyes settings and turned off location detection or whatever that first setting was about the location and I got 2 images describe with people in it, but the other ones were blocked. It’s not fair to us to have to deal with that privacy issue because we’re blind and we just wanna have access to people’s facial details like everyone else who can see and this app would provide that in amazing detail but that privacy issue is not our fault and being blind is not our fault, but hopefully they make this accommodating to us so we can have a reasonable way to connect with people and have that info. It makes for a good conversation when we have that info like this…
The picture you've shared is of a young boy, possibly in a school photo. He has a warm smile and is wearing a dark blue sweater over a white collared shirt. The background is a simple gradient, mostly bluish-grey. The boy has short hair and a cheerful expression on his face. There is also a small object in the bottom left corner that looks like a clip or pin, but it's not very clear.

I tried disabling location services, switching off wifi, and even changing my location using a vpn but I'm getting the same blocked images situation.

I’m not surprised it didn’t work by doing those since I mentioned how it’s not consistent on which photos with people in it wants to describe. It’s discrimination.

you guys are hilarious. you think the face policy is inconsistent, when in fact faces have never fully described or accessible. on a good day, it may describe something about the face, but it is never complete. next time when the ai is in a good mood and willing to describe the face, ask the following questions if they are not already answered:

o describe the eyebrows, are they trimmed and well proportioned?
o does she have double eyelid?
o describe her nose.
o describe her ears, are they symmetric?
o does she have full lips, is she showing her teeth?
o how would you describe her chin, is it pointed?
o complexion of her skin?

even with a perfect picture, the ai will not able to answer all of the above questions because parts of the face is always obscured.

i am agnostic about what the ai should and should not do, this has nothing to do with your rights, and don't conflate this sith democracy, freedom and all the bs. as far as i'm concerned, democracy simply means campaigning and voting; right to due process, equality, freedom and whatever you think is great be be found in any types of political system. simply because one can vote doesn't mean he is in charge, nor does it guarantee your "universal rights given by god". alibaba's ai specifically mentions "helpful for the visually impaired" and "describing face", so this is purely technology and people behind it, very little to do with those abstract ideology.

what i really want is i don't want to see people get super excited then super frustrated about something because they perceive something has changed. at least for the facial algorithm, it is very consistent since it opened to the general users.

All hail our AI Overlords. 🤖🧎🏻‍♂️🧎🏻‍♀️

Be My Eyes, version 5.0.1 has been released on the App Store, but we still cannot describe photos that contain people or faces

A little over 2 weeks ago, be my AI didn’t allow facial features descriptions but a few days later it did but went back again to different messages about privacy…
Now if you listened to this podcast above, clearly facial features are described by the AI & on YouTube the living blindfully channel did a demo a few months back with the same type of full detailed of facial features so your statement was incorrect on the consistency.
A little over a week ago, I was able to ask the AI certain questions like you had above, and it answered me. 😊

Here is an example of how it was.
https://youtu.be/U_UDGFUSGwo?si=bBuxcwhsD8n92BE2

What should we do to solve this problem?

There’s not a huge amount to be done. BME and Open AI are working on a permanent solution which they are hoping to have working before general release. They seem pretty confident this is going to happen. They are both extremely committed to getting facial descriptions working but the law is a difficult thing to work around. They believe they have a solution which is geographically isolate the places where legislation disallows facial descriptions so the rest of the world is able to use them but this isn’t something that can be reliably implemented overnight and it has to be reliable as its compliance with the law and that could lead to legal action or fines. I’m extremely reassured that the best people possible are on this and they will find a solution. I know some of them and their commitment to this issue and the community in general is beyond question.

So now all we can do is hope and pray. Hopefully this program will run well, and when it is released to the public later, we can take advantage of all the features in be my ai better. Thank you andy.

Be My Eyes just published an update on the subject of faces:
https://www.bemyeyes.com/blog/image-to-text-ai-and-blurred-faces-whats-going-on

Appreciate that notice. Just here to share.
https://www.bemyeyes.com/blog/image-to-text-ai-and-blurred-faces-whats-going-on?fbclid=IwAR0jH7cCOh8g60SRZ35PdSC2h8IvGAThz3dXpcIxAFAVD_QqQs6MjsGx_Vo_aem_AXcUchnYOdkaoQ3YJJqk5S9B-yeN_d7Fvc7om3fMe_7I1GKU9Dg_RLzIoALChD3QEpo

The only thing we can do is keep pressing on the issue that those privacy laws are invalid for our personal reasons of having this technology. Clearly, be my eyes is in developing this feature for us, but they know we are frustrated and need the same type of information that everyone else has who can see and has eyesight. They read these comments and they know that we are dealing with this issue personally. The more we make a fuss about it, the more they hear us. We sent them emails, we talk about it on here and they know, they’re working on it. That’s all we can do because laws are complicated.
We have to fight for our rights… to party!

Hi again everyone. Subject says it all. I just opened up Be My Eyes and read the blog post from them regarding the facial recognition. Tbh I don't really care about this for now as I'm rather new to the app anyway, but thank you BME team for clarifying things. As has been stated, this is a complicated issue and it will eventually be resolved. It will certainly take time, but rest assured we have not been forgotten. I for one am incredibly grateful for this app. I haven't done a whole lot with it yet, but that will definitely change in the near future as I *hope* to get a phone with better battery life and memory. I've shown Be My Eyes to at least one neighbor who is very impressed. Another neighbor knows about the app but has never seen it.

I love using be my AI. I think it's such a useful feature, and be my eyes have done a great job with this feature. I've been testing the feature for a few weeks, and I can truly say, it has changed my life. Whilst it can't tell you about faces, everything I've used it for, the accuracy of it has been amazing. This feature has enabled me to read the game menu on my WWE game on PS5, witch before using this feature, I'd always have to wait for sighted assistance. With this feature, I'm now able to do it on my own. Great podcast, and thank you for making the podcast

I'm experiencing the exact same thing, I take a picture of my face, I've even shared a picture from Prince Harry, because I follow Prince Harry on Facebook. And I've tried getting it to describe the picture, and at first I thought it had something to do with what Prince Harry was sending, I literally thought, make no mistake. I literally thought, Prince Harry was showing his nudes. But then, when other people took pictures of their faces, or I did, that's when it give me that sensitive content thing as well. To me, it feels like somebody just came in my sleep and gouge my eyes out.

When this first came out, I enjoyed taking pictures. My boyfriend and I are both blind, and we were both sending pictures of each other, he would send pictures of his adorable haircut to me, I would send pictures of myself at the beach. It was a great time, like every couple sent selfies of themselves, that's what we were doing, and it described everything accurately. But for this to happen, it feels like our eyes have been pulled out of our socket. That's how I see it. I really really do hope they fix this. I'll tell you something, at first I was supposed to photos and I was supposed to be my LA.
I meant, I was very opposed to this, be my AI. I was opposed, and I disagreed with it entirely, but then, when I signed up, and I started taking pictures, it was a game changer, and I all of a sudden apologized for my horrible opinions against it. It really not only, changed my life, it enhanced our relationship. Only for this to happen, it's a disappointment.

The similar thing happened to me. Yesterday I try to send a picture of a funnel cake come up to my boyfriend to show him what it looks like, and I could finally say what it looks like, as a blind person. But I try to show him, because, they don't have funnel cake in the UK, so I tried, showing him what it looks like, and, when I took a picture of it, it gave me that sensitive content message, however, when I open the picture on WhatsApp that I sent to my boyfriend, it described it very well. And he even got a description of it very well, so to my relief, he was able to get an idea of the funnel cake. But it's just so weird, how in one app, it'll tell you one thing, but then it tells you that it doesn't work, or this whole sensitive content image blocking placeholder thing.
I really hope they fix this, because this is seriously annoying

I think, as of yesterday to me, the message changed as well, saying that they couldn't tell me because of faces, but the image was blocked, because I detected a face or something like that. I still don't see nothing wrong with faces at all, yeah, the message must be better than the previous one, and it's understandable, but I really feel the persons face shouldn't have to be viewed as sensitive content, if it was something else, and that's understandable. But it said that similar thing to me yesterday, when I was trying to send a picture of a funnel cake, or a person holding the funnel cake. Which was me.

Tre,
You put it very well. It's like we were granted our abilities to see then abruptly, it was taken away.
It is very jarring and I also derived tremendous pleasure from being able to appreciate photos. My wife and I were having so much fun going through our old photos of our marriage, honeymoon and vacations together.

I think this blog post raises a good point. There is a difference between descriptions and identification. at the moment it seems like those lines are a bit blurred (pun absolutely intended). This is what gives me hope that it will eventually return as it was before.

From my understanding it's a battle between be my eyes and gpt chat4, gpt doesn't want it while be my eyes does.

Just now I try to recognise a Photos that has people on it using Be My Eyes. I used two different photo and the result is good. There is no such thing like placeholder or something. Maybe you can do the same thing like me and post the update and result here. Thank you
And this is my example
The picture shows a group of people standing in front of a restaurant at night. There are five people in the group; two adults and three children. The adult on the left is wearing a white t-shirt and the one on the right is wearing a teal headscarf and a dark-colored shirt. The children are in the middle; one is wearing a maroon shirt, another is wearing a light blue dress, and the third is wearing a red shirt with a grey jacket. Behind them, there is a sign that reads "Gacoan Mie Pedas No. 1" in bright neon lights. There are also cars parked in the background and a "Buka 24 Jam" sign which means "Open 24 hours". The picture has a caption that says "Late post" and there is a timestamp in the top left corner that reads "17.13".

It looks like Be My Eyes have a bypass with open AI and face descriptions should work. Hopefully they will stay this time.

I can confirm that it appears to be working fine here for the time being. I echo the sentiment of the previous poster. I hope this is the long term solution.

Yes, they fixed it!

If they take this away again, I am not going to even attempt to use the app anymore the back-and-forth with us is a crying shame for the company to do that for us and then take it away again. I’m grateful that it’s back but again, if this happens again, I won’t even attempt to do anything with the app with people in photos because I know it’s going to put an upset with me for the reasons that are already out there!!!
It did a wonderful job at telling me what I needed to know from an image on Facebook dating not long ago and want that to stay just like everyone else and it’s not like we don’t have good reasons for that not to stay so we can get the information we need, have good conversations with people, and enhance the quality of our lives by having that information. Thank you for whoever made that happen, but please do not take that away again. I appreciate this. 🙏
Now, to address the people in the comments, yes, when we have the information about peoples faces, we are able to converse with people about relevant information regarding their looks, their facial expressions, clothing or whatever information that is gathered in that photo that we can talk about freely just like anyone else who can see faces Which makes our lives that much more better. And it’s not just about the faces, it’s about the clothes that the people are wearing and the activities they may be doing in that photo. Seriously though, why would someone want to block that from us? If the shoe were on the other foot and sighted people suddenly became blind, and they wanted to have that information, they would feel the exact same way we feel about this technology. It makes absolutely no sense for anyone to block that from blind people having that information to use in a positive manner. And that brings me back to laws, it makes absolutely no difference when it comes to laws because it’s not like we are using that information to do any harm to people or whatever the case might be.

It warms my spirit to see that people are excited about this feature being added back, and when the person above explained the situation with her boyfriend who is in another country, they sharing their lives back-and-forth with the descriptions from be my AI, and the gentleman above explaining going through photos of his wedding, and having that conversation with his wife about their wedding photos, that just made me so ecstatic for their relationship and the mini conversations they will have With this technology. And it brings me back to the reasons why we should be able to have this feature stay and what it means for the quality of our lives. This is not just about blind people talking about images, but what we can do whenever we are out in public, we need to take photos of places so we can get information about whatever we’re trying to find, and in those cases, people might be in that image, and we won’t be able to have that detailed description because of this blockage of faces that happened and that would definitely make our lives harder for us to do without having this technology and not all the time we want to call up a cited volunteer on Be My Eyes to get that information with the live video… we would independently do it on our own which is much more liberating, even though we are using AI technology. Thanks again.

My invite came through a few weeks ago and just like everyone else I was totally blown away by it. I've used it to help identify which clothes I wanted to wear, but mostly I've just been exploring what it can do.

I got married a couple of weeks ago. The day before we had a small family gathering and I was able to use the virtual assistant to describe a photo and was amazed by the detail. I was able to identify every person in the photo and know what they were wearing. Chat GPT was even able to comment that it looked like a happy family gathering. It got bonus points for calling me young!

(I should say that I can remember what each family member looks like as I only lost my sight comparatively recently, so obviously wouldn't have been able to identify anyone on my own in the same way otherwise)

I was really looking forward to trying this out the next day because weddings involve a lot of photos and I was nervous about feeling left out. After the event I got home and checked some photos and was a little dismayed to find that the content had been blocked. Unlike others I wasn't told that it was due to faces being present. It made me feel immediately back to feeling excluded and unable to take part in the conversation.

However, I tried again the other night and was excited to find out that it was working again. I now finally know what the photo captioned"Wow!" was about and was able to understand a photographic joke that was also posted in. Sure I could have asked my sighted wife to describe the photos for me, but it's not the same thing.

Anyway, this tool has totally blown my mind about what it can offer me. Not so much in specifically useful tasks which I've not done much with, but in social inclusion and mental wellbeing. But it also makes me nervous about how easily it can be taken way.

I'd love to be able to point something like this at a whole photo gallery and have it automatically caption all the images. Particularly if I could train it to recognise individual people.

I'd love the ability to add photos from the library in the ask more area. Additionally, I'd like to eventually use custom prompts and be able to ask a question before sending a photo. For example, I could take a picture of a menu, but immediately have it tell me about items which have prices under a certain amount without having to then go to ask more. So far though, this technology is incredible! I'd used it at the hair salon and it read the posters to me and described the products on the shelves.

if the reply to my question is long, the text is cut in the middel and I am not able to read the whole answer. Is anyone having the same issue?

I bet this doesn’t happen in English. It happens all the time for me in Greek but in English everything’s good. Thing is I can’t change the language from Greek to English. I think it’s not a be my AI problem because it also happens to me with ChatGPT, only I can change the language with this last one and everything works fine.

Where this happens, it’s a technical limit for now with Open AI. There is an easy fix for it. Just ask be my AI to continue. It will carry on the message it cut off.

Hi all. I'm not exactly sure how to say this, but here goes. Just last night I received a reply to a message I had posted on the discussion list for another app with virtually the same functionality as Be My Eyes. I won't give out the name of the app, but they are having a webinar on Thursday which I cannot attend live. But I registered anyway because I'm going to listen to the replay when it becomes available. Here's my question. Are all these OCR/image-recognition apps in competition with each other? For example, does the fact that these apps each have their own bar-code feature influence the other? Or are they learning from one another? I hope I'm making sense here. I love these apps and am so looking forward to the future, but have just been rather curious about this.

Hi, My understanding is there are 2 core technologies Which are being developed by Google and Open AI Then there will be separate apps built on these core technologies and anyone who can get access to the API’s and pay for AI processing will be able to build an app. Those apps will all be in competition with one another. I don’t think we are at the point yet where AI’s learn from one another on the fly but maybe thats closer than we think.

If it's an AI one like this then it might be useful in diffirent situations.

I like the idea of asking a question before sending a photo as well. Perhaps that will be looked at in a future update.

You can kind of do that. I’m not sure if it’s what you’re looking for but once you’ve taken the first photo you can then tap the ask follow-up question button. At that point, you can either just ask a question or tap to add another photo. In that window, you’re able to ask a question that goes up with the photo. Back at the start of the beta cycle that was the only way to do it. The first screen with just take a photo was introduced in order to reduce the number of interactions you need in order to get a photo described. Doing more complex things was then added as an option after simplifying the thing most people will want to do most of the time. I personally prefer it that way. Anything that reduces the number of times I have to swipe and tap in order to do the thing I do most often is a good thing IMO.

I’m really happy this is going to be out for everyone, but I think this underscores the importance of not relying on one company to provide a service like this. There are a lot of promising developments in the open source world, and ultimately I would like to see the technology beyond device and private. Right now servers are required because the models are simply too big to run on device, but that is not going to be the case forever, and I think be my eyes is going to have some serious competition in this space within the next year.

Yeh I agree. Maybe it will take longer than 1 year to get LLm’s running on a phone but definitely in the next 5 years or so although who knows what will be the new exciting thing by then. There’s so much money and research around AI at the moment that innovations are all but certain. Remember though, it’s always going to be the case that a more powerful model will be able to run in the cloud than on your phone. My guess is that Be My Eyes would welcome the competition. Competition means they can’t stand still and one thing I know about them is they are incredibly committed to improving lives. Competition would drive better products so I’m pretty sure they’d be up for it.

A lot of this is still up in the air, and no one has all the answers. With that being said, you can already run some language models locally on iOS, and even more on Mac. Really the issue is ram, Apple silicon is more than powerful enough to handle them. I would be surprised if they don't have an announcement in the space by next year.

So, I have an idea for how this could be implemented. Next to the take picture button, there could be another one which allows you to ask a question before the photo. That way efficiency is not compromised, yet it could be there as an option for anyone who wants it. Additionally, the add picture button within the chat should also allow you to use your library. Of course, these are just ideas. I'm beyond grateful for what we have now, and am eager to see what comes next.

I'm in instagram and facebook. I am no longer seeing be my AI on the share sheet of a photo. This was working perfectly last night and even this morning. Has anyone else noticed this?
Andy, is this a feature be my eyes have removed from their end possibly? Would you be able to ask them?

Right so, I just took a photo and got the following message,
Be My AI failed to respond, reason: The OpenAI account associated with this API key has been deactivated. If you are the developer for this OpenAI app, please check your email for more information. If you are seeing this error while using another app or site, please reach out to them for more help.

I'm going to talk to him later on today so I'll let him know this is going on.

I just tested Be My AI a few times and I am not experiencing this issue, so this seems to have been fixed.