Today Be My Eyes launches on Meta smart glasses

I got the email too. I did notice that it said it would be rolling out in my region which suggests it might not be everywhere yet. I am in the UK.

Also it might take a few days to appear.

I think this is absolutely incredible news.

"This first release provides hands-free access to volunteer calls, but our respective engineers will continue to evaluate and develop new capabilities over the coming months."
Like the subject line says, any speculations?

Maybe if BeMyAI was using the Llama model, but that takes away the whole usefulness point...

I actually think the AI has become pretty good recently. It's certainly a lot better than when it started. If it keeps going in this direction then I personally don't have a problem with it vs the other models.

I would also love some sort of Be My AI in there. But it might be that all we need is to go to the next version of llama and have it phrase the questions in a blind person friendly way.

I would be amazed if Meta were happy supporting ChatGPT but I'm not sure we necessarily need it.

I'm inclined to order a pair, but would love to hear some first-hand observations.

I love the smart glasses (Meta of course) for identifying clothing and objects. For example, we had a strange device in the office tucked in with some video magnifiers. The glasses immediately identified it as an UbiDuo, a device for deaf communication.
Yesterday I sorted through a pile of t-shirts and Meta AI easily described the pictures on each shirts and the writing. I also used it to identify food packaging and give me general scene descriptions. For example at our local library in the children's room where I was doing pet therapy, it told me the carpet had a colorful world map stitched in to it.
Where the glasses don't work as well as Be My AI is in reading signs. On the campus where I work we have many buildings, separated by wide asphalt plazzas. I walked around using Be My Ai, pointing at buildings and having it read me signs. So if I was lost, I would know where I was, just as sighted folks do by looking at signage. (All the buildings are gray and look similar so without signs sighted folks get lost too!)
Anyway, Be My Ai had excellent scene descriptions and automatically read signs. The Meta glasses could not do that. I'd have to query them repeatedly and only got lucky sometimes.
I also have trouble finding a bus stop in the middle of a parking lot; it's not on a sidewalk. Meta AI could tell me where it was, but could not give me enough detail to locate it. Be my AI could.
I hate bugging volunteers when Be My Ai does a good enough job, but I also hate trying to hold my phone, hit the Take picture button and ensure it is positioned properly to identify what I want. I too wish Be My Ai was incorporated in to the glasses.
One workaround the Be My Eyes folks could implement is have a hands-free way of using Be My Ai. So I could say "Hey Siri, use Be My Ai" and it would take a picture. At least then I'd have one hand available to hold an object while I held the phone with another hand, or had the phone on a lanyard.
Another tip I've used with both the glasses and Be my Ai is to have the object I want identified positioned on a piece of black construction paper or a black scarf on an empty surface. But it still involves holding my phone with one hand and tapping "take picture" with the other hand if I'm not using the glasses.

The issue with not reading signs is as frustrating as it is baffling. We know AI can read signs. That this doesn't work flawlessly in the Meta Ray-Bans is a total head-scratcher.

Furthermore, connecting to a human BME volunteer can take some time. It's usually not an issue, but if you're waiting 15 seconds for a volunteer to read a sign for you, then walking 50 meters, then waiting for another volunteer to read the next sign,, it can be very tedious.

As you said, you don't need a human volunteer for this. Be My AI can handle it. But the requirement to point a camera, snap a photo, and wait for processing is a deal killer.

One use I have for the Meta Ray-Ban is to help me with indoor navigation at conferences, to help me find workshop and session rooms. For that, I need a decent sign reading solution. It sounds like I will still be waiting.

So if BME are listening, a couple of other things - proper PCR and integration with Voice Vista or similar please.

Bit disappointing if signs are still not working properly. First thing I did when I got the AI was to try it out on a walking route and it totally failed to read any of the signs at all. I was hoping it might have got better by now but I keep forgetting to try. Of course the other problem is knowing where the signs are.

@paul - maybe a long shot but have you looked at Good Maps? Shaun from Double Tap has been talking it up recently. I think it does require that a building be marked up, but some conferences possibly might be.

Hello Mr. Grieves. I heard the Double Tap coverage of Good Maps. I'm glad people are working on indoor nav. But We've had indoor nav solutions for years that have required instrumenting the physical location, and these systems never attain widespread acceptance. And imagine if three or four do catch on, and we blindies need to familiarize with the idiosyncracies of three or four indoor nav UIs depending on which building we've enmtered. Not good enough, in my opinion.

If a human can navigate a complex environment by looking at overhead and/or wall-mounted signage, then certainly an AI should be able to do the same. If the AI has access to building floor plans, it should be able to do an even better job. I could even imagine someone walking through a facility, taking photos, and training an AI on locations simply based on the camera image, much like we train AIs what a cat looks like. or an elephant.

Sorry. Didn't mean to derail this conversation.

Whilst I agree that manually mapping out a building feels a bit backwards in this day and age, using signs does require that the place you are after is well sign posted. Whereas something like Good Maps presumably doesn't require you to find a sign, and will lead you right to the door rather than the general area. So in theory this sort of thing should be better than what a sighted person has. Just because you can see the signs doesn't mean you can find them and that they make sense as I've found out over the years.

What I don't want to do really is to have to hold my phone out as I'm walking about a busy place. I suppose there's always a lanyard but glasses like these feel like the perfect solution. (I believe Good Maps still requires some sort of camera use?) So, if Be My Eyes and Meta could somehow sort all this out that would be great, thanks! :)

I'm not sure how Glide handles indoor spaces as that would be interesting.

I just woke up and haven't finished my first coffee yet. So, while I'm in dreamland, I'll muse that what we really want is some kind of navigation solution that seamlessly integrates information from a variety of sources to determine our location. View from a head-mounted camera, audio cues, signals from GPS, RFID, or Bluetooth beacons, map data (not just roads but building interiors), text on any available signs, publically available geopositioned images, and even anonymized location data from the navigation systems of other nearby people, cars, or other entities. Such a system would track location as you walk from outdoors to indoors, step aboard an airplane, or take a boat across a lake.

And it's not that different from how humans track their location. They look around, yes, but they use lots of other cues besides eyesight. So give me a nav system that tracks location the same way people track location.

Okay time to wake up. The coffee smells good.

Maybe if BME did use LLAMA it could be better prompted to know a blind person is asking the question, an instruction to read all signage, documents etc. At the moment it is treating each request in the same way as it would for a sighted person. Maybe, at some stage, LLAMA will get memory so we have greater control over the format of responses.

On a side note, has anyone got Audible on their glasses yet? It was announced at the October event but not seeing it in my optional app linkage.

I just checked and now have access to BeMyEyes in the MetaView app. It wasn't there last night at like 11, and my app hasn't updated since maybe Wednesday. So it's clearly something they're rolling out under the hood. Going to give it a test call sometime later today.

I got it straight away but, as I was on the beta, I had to delete both be my eyes and meta view betas before I installed the public versions. I'm not certain this unlocked it, but it may have.

It's probably something I"m a little to shy to use myself, but it's great to know I have it as a back up should I need help in a pinch.

If BME team reaches an agreement to integrate BMAI into meta glasses today, it will be 3 steps up from what we have with meta glasses. For one, for whatever reason, whatever prompt they're using with gpt is way better than any other similar app. And second, the way I understand it, BMAI does seem to have access to facilities in gpt which a regular image description with the gpt app doesn't give. For example, if gpt thinks an image uploaded through the regular app contains something explicit, it is likely going to refuse to describe it. But the same doesn't happen with BMAI, and that's how it should be.
Currently, meta ai refuses to describe people; which would and should not be the case with any image description service for the blind...

As of the past couple of weeks, I am a happy AirPods Pro 2 user. They act as hearing aids. If there is a compatibility issue between the AirPods and any other device, I'm sticking with the AirPods.

I know the Ray-Bans are bone-conducting, so I assume they are physically compatible. The AirPods go in my ears, the Ray-Ban temple pieces rest over my ears.

It's the Bluetooth connection that has me worried. I know for a fact that if my phone is using any other type of Bluetooth audio, that as soon as I put in my AirPods, audio switches to the AirPods.

Will I still be able to use the Meta Ray-Bans if they aren't connected to my phone by Bluetooth? Do I lose all functionality (it's one or the other, Ray-Bans or AirPods), or only lose some functionality? Or would the Ray-Bans work just fine in this case because they would simply route their audio through my AirPods?

Standalone, you can use the Meta Ray-bans to take photos and videos I believe. But nothing else. All the AI goodness comes from the MetaView app on your phone. So I think without bluetooth the glasses aren't going to do much.

Also I don't think the Metas are bone conduction. I believe they just fire the audio at your ears from the arms of the glasses.

Someone more knowledgeable than me may know better of course.

Just to continue brian's tangent of the new features... you can also scan a QR code with the glasses and it sends the link to the app and you can point at a phone number and ask the glasses to call it. Also it will announce the name of an incoming caller.

The reminders thing sounds awesome. E.g. take a photo of your hotel room door and ask meta to remember it for you.

Not all features in all regions.

I do wonder if being in the UK but on a US VPN means that the BME feature may not be unlocked in the same way. I don't think it's available for me yet unless I'm looking in the wrong place.

You can use both the meta ray-bans and airpods pro 2 however, you will have to have the airpods pro in transparency mode as all the meta interaction remains on the glasses. It would be great if it could pipe directly into the airpods but it doesn't.

A quick tip, if you go into settings, bluetooth and long press your metas, you can change the type of output it is so, instead of audio from your phone coming through your ray-bans with the associated lag, you can switch the ray-bans to speaker, and voiceover will come from the phone instead and then, as above, you can use AirPods when you are out and about, if you like.

It's an option at least.

No, the meta isn't exactly bone conduction. It just releases the audio close enough to your ear such that you can hear it without having to use an in-ear earpiece.

Okay, thanks for the clarification regarding bone conduction. I had that wrong. But it sounds like the take away is that the two devices are ear-compatible. Using AirPods doesn't physically prevent the use of the Ray-Bans.

The more important question is Bluetooth. How useful will the Ray-Ban's be if the iPhone only connects to one Bluetooth headset, and it's the AirPods? It sounds like the Ray-Bans might have some use, but not full functionality.

If I put the AirPods in, which breaks the Ray-Ban Bluetooth connection, then take the AirPods out and stow them, how easy is it to restore the Ray-Ban Bluetooth connection?

The iPhone does connect to multiple bluetooth devices at the same time, but I haven't personally tried connecting the meta glasses along with any other device so cannot say how much of the functionality will be available...

I know several IOS users who got access and are using it but none of the Android users I know have gotten access yet. FYI, they are all running the latest versions of BME and Meta View, according to their Play Store, as well as the latest firmware available for their glasses.

I just read this comment on Mastodon:

Post DAVID WOODBRIDGE, ‪@Woody@dragonscave.space‬, “By accident found that I can still use my Airpods on my iphone for VoiceOver, Siri etc, plus keep using the Ray Ban Meta Glasses at the same time. Assuming MetaView is controlling audio to the glasses separate to OS Bt connection to the Airpods.”,

I had a try with my air pods 3 yesterday and thought I was experiencing this too but I got a little confused about where the sound was coming from. I did try a second time and once the glasses took over I couldn’t get it to revert back to the air pods.I’ll try to find some time to play about with it some more when my brain isn’t feeling quite so frazzled.

Nice to have such AI features like Be My Eyes and others on the Meta Glasses.

Question: I know Be My Eyes is free to use, but if you are using other AI features of the Meta Glasses does that require a subscdription?

--Pete

I'm planning to try these Thursday at Lenscrafters, a large retail chain that sells them here in the US. I will at least pick out style and size.

I'd love to give them a real trial run before I buy. Do you think they would let me setup their floor model with my phone? What kind of things do you wish you'd tried before you bought? Thanks.

...and what features are available using the Meta View Software? From this thread it sounds like you can query AI for descriptions of what is around you, have it OCR text, etc. Are those services free?

Also, it sounds like the device pairs with your phone if I udnerstand this right. Thus, I assume you use the glasses as the camera to your phone and get speech feedback from your phone sent to the glasses? Is that right?

--Pete

I for my life cannot get this Shazam-kind of thing to work. any prompt for that?

I thought the feature to tell you what was playing was limited to the music playing through the glasses, as opposed to being a general Shazam thing that can tell you about whatever music is playing in the background. And in that case it probably knows what is playing without having to figure it out by listening.

I might well be wrong, though - I think this is just what I presumed from reading the release notes for the last upgrade.

And, apologies for going on-topic for a moment :) but the accessibility option turned up for me yesterday. As mentioned before I don't think this coincides with an app update, but I did need to close and reopen the app for it to appear.

When Be My Eyes is available, you configure it through the MetaView Settings. A new category "accessibility" appears alongside the others (under Communication I think)

I may be wrong about the music recognition thing. The release notes appear, I update then can't figure out how to get back to them to check. Also a lot of the new features were region locked if that makes a difference? Anyway it didn't work like Shazam when I was just listening to music in the houes the other day.

OK I had another go with this and it does seem to work. I connected my Air Pods 3s to my phone and voiceover came out of them. I then put on my meta ray-bans and could speak to the AI there. At the same time, VoiceOver continued to come out of the Air Pods. I could also talk to Siri and she would come through the air pods too. It worked surprisingly well.

I can't test with things like transparency mode or using them as hearing aids but I'm more confident that they might work now than I was before. How well they work over time I also don't know.

I'm hoping to get some Air Pods Pro 2's at some point but won't be for a while. I'm sure someone else on here must have both that can be more helpful.

"Meta versus AirPods"--I like it. Could be the name of the latest Kaiju movie. I'm looking forward to visiting lenscrafters and trying out the Meta glasses in a couple of days. I'm installing the Metaview app, hoping I'll be able to pair their demo model.

I watch a video on youtube, it is talking about we can do live conversation wiht gpt 4O using be my eyes...
anyone know that when will this feature released ?
https://www.youtube.com/watch?v=KwNUJ69RbwY

Hi Brian,
Jonathan Mosen did an interview on his Access on podcast with Mike Buckly, and he asked about the video feature and Be My Eyes. Mike Buckley said that if he had known Open AI would have taken this long to release the video feature, he wouldn't have released that demo. But I actually thought Open AI released it, which means Be My Eyes wouldn't have had any control over it surely?