Enabling Look and ask with Meta AI on Ray-Ban Meta smart glasses

It does say that it is only open to a limited number of users, and also it's US only. So maybe just need to wait a bit and keep trying?

I hope they open this up to the UK as well soon as I'm really curious to try it out. Not that I have a pair yet.

If anyone does get onto the early access program, please do let us know what it's like.

Hey folks, I got a pair of Meta Ray-bans a week ago and just today got access to the "look and ask" preview / beta.
I tried the same gestures as you did Kevin and they didn't work. I had my wife look and nothing. So apparently it only becomes possible once you get the invite.
Today, I turned on the glasses, put them on, and started the app, and there it was.

It took a few steps. First it wanted me to agree to some stuff, and then update the software and the glasses software.
At first I didn't know how to use it.
You've got to say, "Hey Meta, look and..." And then tell it what you want it to do.
So, for example. "Hey Meta, look and describe my surroundings."

It then takes a picture and you hear a short series of tones and it describes.
You can then ask it questions.
Two major points to make here.
First of all, it definitely hallucinates. Or maybe it's better to say that it seems to imagine it sees certain things when it doesn't. Or it confidently assumes or infers. Earlier it saw a bow and arrows hanging on my wall when there is no such thing.

Also, it omits or ignores information that it thinks might be private or privileged information. It will not describe people, not even the color of a shirt if you ask it. But, interestingly enough, it might include it in the description.
Also, it won't read what it thinks might be private information.
It will read handwriting, at least in my experience.

This is my experience so far in the last couple of hours that I've been playing with it.

Seeing ai is better for describing people.

It's great, but limits descriptions of people and faces and stuff like that. I wish it didn't but what can you do?

OK, with a quick google search it isn't using chat gpt but its own version. I wonder why this is a feature then? Is it for the blind or is it a novalty thing?

Some mainstream examples they give are to help people pick out clothes that match, help identify plants, help come up with recipe suggestions from a photo of ingredients in the refrigerator.

Also scientists are using similar tech to get ai input on images of work they are doing. technicians can get ai help on things they are working on, identifying engine parts or what ever.
It's a blessing to us. We are, in a way, a biproduct of people wanting visual help for themselves.

Ok; that's awesome!

The voice is nice,, and if it were able to play podcasts and guide me as a blind person I'd probably buy it over the apple vision pros.

Tech is getting exciting again.

Hey Brad, If you want to listen to podcasts or music, these are GREAT also, There are very discreet speakers in the arms of the glasses which allow you to hear not only the voice, but also the VO voice and music or podcasts.
About guiding you down the street, well, I don't know about that.

Will the apple vision pros. do that?

Do I need to save up for a pair of those? :)

Yeah, this is the most excited I've been about tech in a while.

Hi! These glasses seem really interesting to me.

For those who have already tried them, are the responses from the AI as fast as shown in the reviews?

Best regards.

I asked the glasses to video call my wife and it just did an audio only call. I had a quick rummage around the help and couldn't immediately find a way to do a video call. So it's possible I've missed it, but first reaction is that you probably can't.

Re podcasting, they act as normal bluetooth speakers too so you can listen to podcasts via your phone. As far as I know you can't explicitly ask it to play a particular podcast. You can tap the frame to magic tap if you want to pause and resume, and you can swipe up and down to change the volume. It's a nice way to listen to them.

I'll leave it for now,, I need something that can guide me around.

AIRA is great for navigating indoors but outside, they can't help you cross a street. Your mobility skills have to be up to par. They might be able to tell you if a car's backing up you didn't see but other then that, you're on your own. Think how many people would sue saying well, so and so told me it was safe, but I got run over by the bus. i'll have a good experience with aira when I head on vacation in a few months.

Hey everyone. I got the glasses a few weeks ago and I have used them quite a lot during this time. I live in Norway but have used a proxy to enable AI functionality when first setting up the glasses. I cannot enable early access yet because I did not Get invited. Will wait for invitation or hopefully for a full rollout soon.
I am totally blind but still have some light sensitivity so I appreciate the adaptive lenses. The speakers are more than OK for indoor use and even for outside if it is not too noisy. My wife tells me that the camera takes good pictures and videos.
I sent feedback asking that we get the option to use Apple Music instead of Spotify and also to get support for direct streaming in meta apps like WhatsApp and Facebook messenger.
The set up process was quite accessible with voiceover, the glasses feel very comfortable for extended periods of time, the carrying case is premium, but the only negative thing that I found is that the voiceover is quite laggy when using the glasses.
Please feel free to ask if you have any questions about the glasses. Wishing everyone a fantastic day ahead.

The glasses are bluetooth. They do have wifi in them - you can add your network to allow them to automatically update their firmware, but as far as I know that's the only reason for doing it. Also when you transfer photos and videos from glasses to phone, it seems to start a wifi network on the glasses that the phone connects to - at least that's what it sounds like.

I'm not aware of any multipoint ability - not sure how I would know?

I think you can live stream video to a couple of the other meta social media platforms like instagram. But it might buffer up the videos first which maybe doesn't work for a live call. But this isn't something I would do.

I did also notice that it is a tiny bit laggy with VoiceOver - nothing that bothers me and I wanted to try something else just to compare in case it was my imagination.

Hey Victor, Make sure you have gone in to glasses and privacy and allowed it to save your voice recordings and transcripts. I understand that is one key to possibly receiving the early access invite.
Also, I read somewhere that completely closing the app, and turning off your glasses and then restarting the app and turning on glasses might create a situation where you might sortof trick it in to giving you the invite.

I don't know whether this is true, but I do know that it was after one of these attempts that I suddenly got the invite.

I haven't tried this yet, but maybe I should.

As I understand it, it is possible to hear comments made on your video stream through the glasses. So they must expect it to be relatively real-time if that's possible.
I wonder if the wifi connection is used when streaming video.

The WIFI is used to transfer media from the glasses to the phone. But, interestingly, when making a "look and tell" request, the photo must be going to the phone, and yet no request to join the WIFI is made.

I read somewhere, though I can't find it now, that in the camera app on the phone a button comes up allowing the user to choose to take pics with the glasses.

I'm a little concerned that Apple is a bit more restrictive in this area.

One other annoying detail is that it is possible to ask the glasses to take a picture and send it via text, but it requires you to approve it on the phone and I haven't found that process to be very straight forward.

In my experience, VO is a little laggy, but I get that with other bluetooth accessories.
But it's not just vo that is laggy. It takes a disconcerting amount of time for Siri to respond when longpressing the side button. I actually don't have her enabled for voice response but maybe I'll try that.

In my opinion these make an excellent, if expensive, pair of bt headphones because they're open. I can listen to my phone without others knowing or hearing.

So, while not everything is accessible using the glasses voice assistant, the glasses improve my experience while using the phone.

OK, stupid question time.... where is the power button? I googled and it says it's on the left arm near the hinge but I can't find anything that feels like it wants to move. Is it the nobbly bit and do I need to give it a bit more elbow grease?

We are assuming that the reason it's not been implemented is for bandwidth reasons. It might be that there's just no call for it outside our community. Normally people would want their face on a voice call I would think.

Having the option to take a photo using the glasses from the camera also feels a bit weird. Isn't the point that you can take photos hands-free? But if you can then obviously that suggests bandwidth isn't a problem. I'll have a prod about later and see if I can find it, but I'd be amazed if it's there on an iPhone.

If I ask Meta to send a photo to my wife, it does so using WhatsApp and I just need to say Yes when it asks me to confirm. I don't need to use my phone. I guess iMessage or the Phone itself is a bit different.

Don't the envision glasses allow you to stream to an Aira agent? Is that not Bluetooth?

I don't really know what Meta is like as a company with regards accessibility. Do they know we exist?

Ah, America, where you can sue anyone for anything, truly; the land of the free.

I'm really not a fan of that way of running things if you can't tell. This just really proves how money hungry America as a country is.

Anyway, ira isn't for me, it's expencive and I think I read that they don't send you the glasses if you're in the UK, or something like that...

Also, just so you know, you've put then instead of than, I understand it's hard to hear sometimes so I thought I'd let you know.

Get a guide dog. 😏

I'm not a huge animal person but as soon as a robot guide dog comes out; I'll definetly be looking into it.

Subject.

I'm a robot but don't tell anyone.

Hey everyone. So glad this subject is of interest here.
So in the help and support section under settings we have the option to submit a report, that’s where I wrote a few things and sent to them.
I tried all the tricks with VPN, turning on and off permissions, the only thing I didn’t try is to power off the glasses, which I still need to figure out.
Hope everyone is doing well.

So, I managed to find the power off button. It is on the left arm inside of it, exactly on the other side from the logo. You just need to put your finger on it and slide it from the lens towards the end of the arm. Hope that helps.
To answer the other question, I believe that the functionality to send reports is only available if you have the glasses.

Ah - there it is! Thanks. It still took me a little bit to find it for some reason but I've got it now. Thank you.

I finally got round to setting up the wifi on the glasses. As well as automatic updates, the prompt does also state it can import photos and videos whilst charging and on wifi.

I think it will only use this network i the glasses are charging and you have auto-update or auto import switched on. I initially didn't, and tried taking a photo before putting them onto charge. A minute later my watch pinged and asked if I wanted to import. I said yes but when I looked at my phone it was asking to join the glasses wifi network. I said yes but then it didn't seem to import it. Also when I told the app to update my glasses when I was wearing them it still wanted to use the glasses wifi network. Anyway I've switched both options on now. Not particularly important but a bit curious.

One thing I was worried about was that someone could shout Hey Meta at me and trigger commands. I'm no doubting this is possible, so I asked my wife who was sitting a couple of of feet away to try talking to my glasses. It seemed to know that she wasn't wearing them and didn't respond. So I guess it can tell. No doubt if someone shouts right in your face it might not know, but that put my mind at rest.

There are also options in the app to do some sort of session thing - I think this is where you enable it only if you start a session from your phone first. Or you can turn it off and use a tap gesture on the arm touchpad to do it instead.

Subject.

Just in case this question is still valid, the power switch is a round raised bit just in front of the left hinge. In my experience, it's best to navigate it with a fingernail. It feels like a circular area with two levels. The top level is the actual sliding switch. It doesn't really seem to click.
Slid forward toward the hinge it is on. Back is obviously off.

When turning them on, it takes a while, maybe 5 or 6 seconds before you hear anything.

Hope that helps.

Yeah, I was hoping to explore the app before I got my glasses but no. It wants to take you through the setup process before letting you in to the app proper.

These really are surprisingly fantastic. I am not on the Look and ask beta yet but i’m incredibly excited about where this is going. I honestly don’t want to take them off even to charge them. Slightly laggy with VO but no more than other standard BT headphones. Maybe a little more latency than the Airpods Pro 2’s. Still usable though. Just incredible for listening to audio, making calls, capturing photo’s and video and they look pretty much exactly the same as standard wayfarer glasses from Ray Ban. If it’s not obvious, I’m very very impressed with these. Having thrown my trust behind Humane, after their response to requests for accessibility I’m switching my hopes to the Meta Ray Ban. They’re here now, they work great in so many different ways and I’m sure even better stuff is in the works for them. Setup was completely accessible too.

Yep, exactly. They aren’t blindness products and I love that. I am completely blind but hard as I think, i don’t think I own a single blindness product. I just prefer to find and use everyone products to get things done. The R1 is definitely interesting but I think the promise of it maybe isn’t in the gadget itself but the technology of Large Action Models. Who knows where this is all going but if LLM’s are going to be able to affect the world they are going to need to take actions so LAM’s are going to be a critical part of whatever is being built. IMHO. The Ray Bans are even better than not blindness products, they’re genuinely good looking sunglasses. Not a 14 year olds science project and I love that about them. I’m bursting with excitement about where these are going in the next few months. They’d be perfect for connecting to one of the remote assistance/AI apps. The Seleste glasses definitely also look interesting but I believe they’re $50 a month which I think I’ll pass on unless they are truly revolutionary and give me a audio described life which I don’t think they are even close to. From what I’ve seen, they’re a slightly worse version of Be My AI in a pair of glasses. Good definitely but I’ll just hold my phone for $50 a month until the Meta Ray Bans fulfill their destiny. lol.

Does anyone know if these could be used with Be My AI or Seeing AI to read printed text? That would be a game changer for me. Especially if they don't look weird, which I'm hearing here that they don't.

I tried a quick demo of Envision glasses, and they didn't really work all that well, but maybe it was user related.

At this point no agreements have been made to get those capabilities into the Ray Bans but I’m sure people are working on it. Let’s keep fingers crossed.

I'm honestly so excited to see where this stuff goes.

I'm actually going to be talking to the maker of seleste in a couple days. They're planning on adding a map functionality so i'm very excited about that.

They say the dream of seleste is to get it so that anyone can use them. They want to be very hands off in the sense that they want less scrolling on the phone and more interaction with the world.

The glasses cost $100 canadian, then $50 a month. I don't mind paying that at all because you're garenteed product updates, I think there's at least 2 either every week or every month, hardware updates for free,, ok you're paying 50 a month but it would be a lot higher if this wasn't the moddle they went with, I believe there's a mailing list, and a number you can call to get directly to the owner of the company.

As you can tell; I'm also very excited about these glasses.

Yes they're more of a blindness product but they'll be the only one I own and who knows, sighted people might start using these too.

@charlotte, oh you're from the UK too? :) That's cool, I think we've got, me, I think there's maybe 4 or 5 of us.

Anyway, I'd highly recommend getting a refund if you can, I've tried contacting the rabbit team about accessibility and heard nothing back, this was a couple of months ago, they're very expencive so if you can; I think you should get a refund ASAP before they either go out of business or you get a product you can't use.

@Andy lane, correct me if i'm wrong but if these aren't blind products, I don't see a reason as to why they'd have blindness stuff put in them? They might, I think seleste is doing a similar thing, but they are a blindness thing, at least for now, so to me that makes sense.

What else have you found?

The company seams very honest and up front about the costs of things and they even say that if we aren't buying their stuff, they're going to go under and whilst that's obvious, you don't get companies being that transparent that often.

I'll let you guys know how my talk goes once it's done.

One small thing I've noticed with these glasses is that the max volume level doesn't seem that high. I was listening to WikiTrip in the car with these and it was a bit of a struggle to hear what was being said over the sound of the car engine. Just about possible but I was trying to crank the volume up and I think I hit the limit. I still haven't been out and about properly with them, so looking forward to hearing how they cope with general urban noise. I think car engines can be particularly tricky with sound anyway. Could be just my ears.

Other random thing - I was listening to Double Tap a couple of weeks ago and they were talking about these glasses. Apparently when you record videos it uses spatial audio which is pretty neat. But I think they also said that video clips were a max of 60 seconds in length, but I've not quite had the time to double-check as I'm not sure I remembered correctly. I need to try recording a few things, just life has been getting in the way a bit recently.

Well I tried a little video of the dogs running about. Apparently I got more of my wife's backside than the actual dogs, but I did manage to get a short clip of them. But it sounds great - really getting good directional sound. It was pretty impressive. I think there is also a lot higher chance of me getting something I want on camera given I just need to point my head and not the camera on my phone which has a much higher chance of going wrong. The fact I managed to capture a little slice of family life was rather nice.

The seleste glasses don't have the best sound iether, from what i'm told and they're going to be improved,, but, this will probably always be a downside to audio on glasses.

The sound for the Meta Ray-bans is very good, the only time I've had any problems was in the car. Our car is a bit noisy at the moment. It's never been the same since I stuck my cane out the driver window and went for a spin.

(Obviously only kidding, but there is some sort of fault that is making it a bit noisier than usual, but not sure how much a difference that makes.)

But they have a lovely sound otherwise - when I was out yesterday doing the video it was all very clear. I had my phone in my pocket the wrong way up and it kept announcing the time every few seconds which was very clear! My wife commented on how good the sound on the video was compared to the one she tried to shoot on her OnePlus a month or so ago and all you could hear was the wind. In my one you could hear us really clearly, and the squeaky ball, dog noises, the birds in the trees and so on. It was like being there again.

I've listened to a few podcasts and things at home and they sound great. And I find them more comfortable than my Air Pods 3.

One thing I noticed yesterday - if you stick your hands over your ears, maybe touching the glasses arm slightly, the sound can get really loud. So I think if you were in a busy street and struggling to hear a specific thing you might be able to work around it. Obviously not quite hands free if you have to do that.

Anyway would be interested if someone has taken there's to the streets. I think they would be fine in normal surroundings, so only likely to struggle if it's very loud out where you are.

I wasn't really expecting this, but I'm quite enjoying taking little videos with these things. It's occurred to me recently that almost all my memories are in photos I can no longer see, and I don't have a great mind's eye so I can't really recall much. One of my dog's isn't very well and it's been really upsetting to me that I am struggling to remember what she looks like. But the other day she was sitting in the dog bed making silly noises and I put the glasses on and made a little video. Now I have 60 seconds of her voice and it's really nice. But because I was using the glasses it works visually too. Whereas if it was my phone I think it would end up being all over the place. Not to mention, holding a phone out is a guaranteed way to stop a dog messing about, and it just seems so convenient to ask Meta to start a video. It's occurred to me that having little audio snippets is probably going to be something I might appreciate in the future. Now if I can just get her squeaking her ball or noisily eating biscuits. And as mentioned before, the audio quality is excellent.

Videos are definitely restricted to 60 seconds though, and you can restrict further if you like. But actually that works quite well - you can tell it to start recording and then not worry about stopping it. Or you can use the button or ask Meta to do it if you prefer.

I finally had a very short visit to town with them. They sound great in the car when it's not on the motorway, and on a street that wasn't too busy I could easily hear BlindSquare. So think my initial concerns were likely unfounded. But I'm hardly going into a busy city centre at the moment.

Still itching for look and ask though. Hope it comes to the UK soon.

Did anyone else get a glasses update the other day? Over the weekend it updated to 2.1, but when I asked it what was new it just gave some kind of "unknown version" "don't know" type of page. Not sure if this was because I wasn't connected to the glasses at the time, but not sure what has changed.

I did notice you can ask Meta to send the last photo you sent with WhatsApp which I'm not sure was there before. (As opposed to asking it to take a photo and send it in one go)

Just recently I got an email from the Early Access program talking about the "look and read" functionality.
I've given it a try and it works!

Just this weekend I went to a local regional park for a walk with the family. I stood in front of one of those big informational signs and asked Meta to look and read. She told me all about the history of the park and its surroundings.
One problem is that she started out by saying, "Yes, here is a summary of the text." But I think she was actually reading. I've filed a complaint.

Also, this morning I asked her to read the information on a check I received from a job. She refused to read giving me a lecture about privacy. But, after a few more tries, she suggested that I ask her for details. So, then I asked her who the check was written to, she responded correctly. I asked her for the invoice information. She was correct with some OCR issues. I asked her the amount of the check and who made it and the date. All correct. So it's coming along!

On another related note. The other day I went to a restaurant. I used my phone camera and took pictures using Seeing AI. The combination of seeing AI and the meta glasses was really superb. I could listen privately and get the information I wanted.

There was a comment above about volume. While they aren't too loud, I find that the clarity helps me hear what I need to hear. Those who have them should also consider that you have control of the volume on the glasses, but also your phone volume, and your speech volume. Sometimes I forget this. So I can generally get enough volume. Unless I'm at a loud concert! :)

I have to say that though music is ok, often I think I hear a bit of low end distortion, doubtless because I'm trying to pump a full range of frequencies through admittedly tiny speakers which are not in my ears. One thing I occasionally do is put one hand over my ear to direct the sound in to my ear if the ambiant sound is really loud.

:)
Rusty

Thanks to the useful comments here, I was able to activate it using vpn look and ask However, after turning off vpn, this feature worked for a while and warns that it will not be available where you are now. I don't live in the US and will I always have to use vpn to be able to use look and Ask?

If they roll out the features in other countries then no, but if they don't then yes.

I hope this isn't like the google maps accessibility features that are availible in the US but not anywere else.

Is anyone in the UK currently able to access Meta AI on Meta Ray-Bans? I’m confused as Google searches throw up posts about people having access without needing a VPN, but I have no related options in my Meta View app.

All hardware and software up to date. Using iOS.

I’ve tried the resetting/unpairing/re-installing the app suggestions, but nothing appears.

The meta.ai site says it’s not available in my country yet.

All very confusing, but would be good to know if it should be working or if I’m wasting my time and just need to be patient.

All advice welcome!

It's free but from what i've heard the accessibility isn't that good on IOS, it's usable on windows though.

If VPN is currently the only way of getting all AI functionality I might just wait for the time being. Don’t know much about VPNs and therefore uneasy from a security POV…

Frustrating though as I bought the glasses because I’d read about people getting the rollout in the uk and they even covered it on the Double Tap podcast.

So after doing some reading around VPN I realised my phone might actually be more secure if I were using one!

So I duly signed up and voilà. Now have fully functional Meta AI.

Exciting times!