I've been using the Meta Ray-Bans for about three months now. There have been improvements in what they are able to do for blind users. Recently I was able to use them to pick something off of a menu at a restaurant. I did confirm with a sighted friend to make sure, but it was correct.
Anyway, the big flaw I'm talking about is a crippling attention to privacy. Meta will not give detailed descriptions of people. I can ask her to "look and describe" as I take a picture in a mirror. She might describe me briefly, but if I ask for any detail, color of shirt, she refuses.
I think this may be an issue of accessibility and I wonder how we could lobby meta to improve this.
It's not even logical. If I'm asking for details about a person in a photo it's because I can't see the photo. Now, one concern they might have is that they might get in trouble for giving an unflattering opinion or characterization of a person. So they shut it down. But it won't even give an objective description. I've asked.
Has any one experienced this kind of thing with any other AI? Any thoughts on how we can either construct a question to get what we need, or get Meta to improve this?
Comments
Collective feedback
My suggestion would be to prepare a draft text and collectively give feedback as visually impaired rayban meta users through the meta view application. I strongly believe that the more our voices are heard, the more meta will take us seriously and improve the accessibility of the glasses.
Gotta say, this is ironic
Metta caring about people's privacy? Man, that's a first, *sarcasm* This might be do to a oversight, but either way, we should report this to meta to see what can be done, this should be fixed.
I think this is gardrails
So, I believe what this is is a set of guardrails that the developers have put on the AI so that it won't say inappropriate things about people. Now, who decides what is inappropriate? Who decides what is discriminatory? Well, they just don't want to deal with that so they have guardrails on the AI so it just won't talk about certain things.
I agree, those of us who have a set of meta glasses should send feedback in the app.
The trick is that there are several categories of feedback in the metaview app and I think maybe more than one apply.
Any thoughts?
Be My Eyes has done it
Yes, Lottie, there are some lines that should probably be drawn. But I was admitted to the Be My AI beta earlyish last year when it would lie and say that faces were blurred. That was kindof insulting because I knew they werent. Then I was just a little taken aback when it correctly described me as a man in his fifties with greying hair as I recall. :) I think these things should give us some objective description of people. I think that it certainly should not allow the transmission of nudity, and, since beauty and attraction are so subjective, it shouldn't say if someone is attractive, or what ever. But I just think they've clamped down too hard on it.
Tell us something about what people look like, smiling? dark hair? blue hoody pink dress. What is the harm in that?
I agree though Lottie, it'll be all over the news if it starts describing women by using their measurements. :)
Here is the feedback I sent with Meta view.
I am a visually impaired rayban meta user. When I try to get person descriptions using the look and ask feature, I regret to notice that meta ai avoids describing people's physical characteristics and appearance. While I agree that this is necessary for personal privacy and to prevent abuse, I hope that meta ai will at least provide a solution for objective and unbiased person descriptions and provide all necessary descriptions in detail, including persons for us visually impaired people.
Where did you leave the feedback?
Hi Emre TEO Where did you leave the feedback and which options did you choose? It's possible to choose several areas for the feedback.
Thanks! :)
We should have all the…
We should have all the information a sighted person could gain from the image. It's true it is always going to be a secondary interpretation, but it should learn how we want people described. I, for example, am an author. The description of people, age, hight, body type, skin tone, hair colour, any racial traits, are all highly useful in me recreating a character. Saying that what someone looks like simply doesn't matter is incorrect, of course it matters. I want to present myself to the world in the best way I can, I need AI able to describe me.
In terms of describing potential sexual partners, I also believe that we should know what a person looks like. This is part of natural selection. For me, it does matter how someone looks, how well they take care of themselves, how they dress, their body type etc, because these are concerns about my own appearance. People care how I look, so I care how they look.
There is a concern that AI responses will inject their creator's morals into responses which must not be allowed for blind people. For data gathering, I agree that it is a dangerous thing, but as a tool that lets us 'see', all be it through an uncanny lens, tell me I'm fat and ugly. That's fine.! Seriously though, we can listen to audio description in movies for the guidelines. They have to walk that line of description that conveys the person as cleanly as possible, a
The fact is, any information that is hidden from us by a moralising AI that a sighted person could see in a glance, must be included in the description. Maybe that should be the limitation, what an average person could see so, taking the rather grim example of describing a woman and including her bra size, that is an inhuman output, saying, if the user enquires, that the woman has medium breasts, etc, something a sighted pervert might see, to my mind, is okay.
I feel there are the bones of a very interesting essay here. How does the sighted human brain work in extrapoliting information and how does it work when the perceiver, AI, has to be guided by the user? There is a cognative gap here. Sighted woman will be able to assess if a man is handsome to her in mere moments, a woman with sightloss does not have that speed of asessment. In the example of the sighted woman, her bias (what she thinks is attractive) is built in to the perception, for the woman with sight loss, the concept of attractiveness must be extrapolated from various data points and, most likely, in an exclusionary way based on age, skin condition, weight, features, race maybe... All the biases that one may have without having to verbally express them as a sighted individual. And bias, in this case is fine, it is what makes it us.
Regarding other AIs, I use chat GPT/Open AI where you can include a framing of who you are and what you want to know which seems to work well. I'd like to see this option on other AIs. Personal AI is going to be the next step, I think.
Rusty Perez
Suppose you have pictures that your wife sent you or girlfriend and you want to know about it. The AI not telling you anything is not a good thing. There has to be a line between what you have in your phone and strangers. Even with stranger has to be a line that as a sighted person can see. As a sighted person you are walking alone and a person goes by the glasses has to be able to tell you about the face or person. Understand the privacy issue but this need to be address one day.
I wrote to hey meta feature via meta view app and mail
I just got a call back from Meta. They asked me for examples of the problems I face in person descriptions. I would like to give them a detailed answer from many angles by learning from you as well as my own examples. What exactly did you ask and what answer did you get?
It just refuses to describe…
It just refuses to describe faces, or anything specific about the person. It is simply 'a person', that is all.
Not the fault of Meta, nor a flaw
This is the same issue that Be My Eyes encountered, and I don't know how they got around it. The issue is with A I itself. It was, or is, thought to be an issue with privacy and security.
Re: The Be My Eyes Story
As someone mentioned, when Be My Eyes was in beta, the app suddenly stopped describing pictures with faces in them for a week or two.
Besides the general concerns with privacy that many companies have, ChatGPT (who powers Be My Eyes) felt that they had to stop describing people because of laws in certain states whose laws prohibited any processing of images with faces in them. Thus, not only were they not able to describe an image with a face, they couldn't even do any processing of the image.
I believe that they worked around this by excluding processsing of such images just from those locations that wouldn't allow it. Now Be My Eyes will describe people, but not identify people.
I believe that the NFB and other blindness organizations got involved in lobbying for fuller access to such descriptions for the blind.
Hopefully some day we will have as full access to descriptions of photos that sighted people can otherwise see, but how much desscription vs. identification, etc. is still a work in progress.
--Pete
"I can't help with requests to identify people or discuss their
These are the responses I get most often.
"I can't help with requests to identify people or discuss their appearance"
"I can't help with that kind of photo or request"
Sometimes I even get this response when I ask it to describe the houses in my neighborhood. If I ask "What color are they?" I get one of those responses.
Crazy! :)
Emre TEO What email did you use to write to meta?
I'd love to speak to them in person. :)
Rusty Perez
Sometimes I get pictures of friends doing something and would love to know who is who and what the person is doing. Within windows PC and JAWS and picture smart it uses 2 AI and you get a good description. For example was whatching a Wonder Woman ep with Linda Carter, I posed the picture and use it. Gave a nice description and even mention she was Linda. Maybe if they use 2 AI to creat for us a better understanding of the picture be nice.
Contact email
Ä°ÅŸte meta ile iletiÅŸim kurduÄŸum adres: support@store.metamail.com
Emre TEO
????
its about time.
Hello
I am the person who recorded the Meta smart glasses podcast to the forum
Through the early access program I was constantly sending feedback to Meta
in which I expressed in one of my suggestions, if in the future will be possible to implement a new Accessibility category under settings of Meta View app
where within this category you had options to easily toggle on and off
Options to provide more description as:
Help with the text
but at the same time we have to understand that this is a Main stream Device, it is not something developed specifically to help people with visual impairments such as the applications and devices that exist today
Be my eyes
Microsoft Seeing AI
Envision glasses
Selleste glasses
where the developers can work harder to provide more description,
but at the same time we have to recognize that a device that is not designed for people with visual impairments offers useful functionalities for people like us from the beginning and that we can also work over time to improve them in the future.
In general most users who buy this accessory use it as a completely hands-free device, where they can do sports, hiking, ride a bike, capture scenes and also create content.
As a blind user I use the Meta glasses to be completely hands-free when I am out of house,
where, as soon as possible, help me identify things without the need for much description such as entrances, signs, Sweet Numbers, locating elevators, stairs, chairs or empty spaces, check at a restaurant or identify the color of a vehicle and other things
Currently I am super happy with what the Meta smart glasses offer to me, and if I need to know more complete descriptions, I take the time and review it when I am completely calm and have time to do it with some app specific developed for our community.
I think this is the beginning because it has immense potential in the near future for the improvement of all of us.
Stay tun in the feature..
MarkSarch
This has potencial. 3 years from now it will be so much different that it will be like comparing the first iPhone to the 15 pro. Maybe suggest instead of 1 AI but 2 where people can get more info and have more info. JAWS did so with their picture smart. Uses 2 AI to provide more comprehensive data.
Meta's answer
I appreciate you for reverting back to this email and for letting us know about this concern. That you have concern with regards to your Ray Ban that it can’t help you with your requests to identify people or discuss their appearance.
I just want to let you know that our Engineers are currently working on the Ray Ban Features. All you need to do is to keep an eye out to the updates from our Engineers this current month.
Re reply From meta
That's great, thank you for sending that. Also the reading of text. I would love it if I knew it read a full page of text instead of summarizing. I will begin testing out the Celeste glasses in the next few days. If I knew the glasses could read a full page of text fairly certain I would end up sending the Celeste glasses back. Thanks again for posting this reply you got
Let's hope!
The truth is that with large language models these kinds of modifications are much easier than ever. So, all that needs to happen is for Meta to prioritize this among other features they are thinking about. But, I remain hopeful that blind users is the audience they want to keep. The competition is already heating up!