An external camera is all I want!

I can't tell you how much I agree with this, and how frustrated I am as I watch the technology world march onward with more and more expensive smart glasses. We truly need nothing more than a pair of camera glasses and an app that can take advantage of them.
One of the reasons for this absence is probably the fact that iPhones don't support external cameras. At least, I can't find a way to make it work. If I connect a capture card or a webcam to my brand new iPhone 15, which finally uses a universal connector instead of lightning, it's still not recognized by the phone. So even if we find a good pair of camera glasses, it can't just be a plug & play device on iOS because there is no driver for it. It will probably work on Android, but Android is still trailing way behind iOS in terms of adoption. So while I would be perfectly willing to carry around an Android device and connect camera glasses to it, many people would not, and so we end up with all-in-one solutions like the Envision glasses.
I still want to find this for myself, but when I do, it won't work on my iPhone and I'm not even sure if apps like Aira on Android will pick up on external devices. I'm still stuck on finding one. It's surprisingly difficult.
You may recall that Aira used to give out Android phones with a pair of camera glasses connected to them. Truthfully, if I could find the glasses they were using, i'd just use those on my own Android phone. But again, most people don't have android phones, and most people who don't have them don't want to learn how to use them. It would be a hard sell.
So, until Apple adds drivers to their devices, we're stuck with the all-in-one solutions or the phones. And maybe the best solution for most of us is still just a chest mount for our phone.

Lotte you’ve nailed it here. I’ve long thought that the simplest and most effective solution to what we want is a wearable camera that can simply take over as the camera for your phone when connected, allowing you to use Seeing AI, Be My Eyes, Aira, or whatever your having yourself from the phone.
Sadly I believe the barrier to this is that Apple do not allow it. Only the camera in the iPhone can act as the camera for your iPhone. As it stands Meta glasses can only work with the Meta app, Envision glasses can only work with the Envision app and so on. I would love this to change.
I am dubious about ideas that the form factor for this wearable necessarily has to be glasses, but that’s a side note.
Dave

Dave this isn't strictly true. Envision use aira as a 3rd party connected to their app. Also Call an ally which is like Facetime where you can use your family to help you. If they are using Aira don't see any reason why they couldn't use other 3rd party apps so maybe that is something in the future they are looking into.

Hi Lee.
Yes you can use Aira with Envision but this is an integration between those two companies. I won’t pretend to know or understand the technicalities of how it works, but I don’t believe it’s quite the same as allowing a third party camera to act as the iPhone camera. I’ve no doubt Apple could allow this, but currently do not. I’m sure Apple have their reasons of course, but for what we want here, it’d be great if it changed,
Dave

Hi Dave, I understand what you are saying. But Aira does work with Envision. So, for example if Envision joined with be my eyes or seeing AI via microsoft you could use those apps with the Envision glasses camera. If they then teamed up with say voice vista you would have 1 complete package. May not be exactly what is required but would work.

Ash Rein posted a thread speculating about cameras in EarPods, and an article speculating about Apple coming out with them, additionally camera glasses and a smart ring. I know nothing because I've been fooled by social media before.

Hi Lee. Ah I see. Yes this is true and would be nice. I’d love Celeste to team up in this way too.
Still though, I’d prefer what I believe Lotte was originally driving at, which would be a simple wearable camera where we are in control of what we use it with, and likely at a much lower cost.
One can dream ☺️
Dave

I wonder if your local university engineering kids can slap something together a lot quicker and less expensive than $4000 for smart glasses. A GoPro and some software hacking could get this system working in less than half a day.
A kid in India built a braille printer with Legos for $100.

Something like this should be easy for a hackathon. Thanks for being brave and trying all of these glasses.

Yes, the hardware is important, but the software is even more important, and as frustrating as it is, I don't think AI is quite ready to be a complete visual assistant in the way you would like just yet.

A question I have for Seleste is what more they are able to do with the technology that exists at the moment. For example, I feel fairly confident in predicting that the capacity for AI to analyse live video—which could be useful for blindness navigation—is only a few months or years away, but it seems to me that a company like Seleste wouldn't be able to introduce functionality like that until the developers of AI models add that functionality to the models.

It would be magical. A camera that also tracked my eye directions and followed my focus would be even more magical. I often shift my auditory focus of attention in concert with eye movements. However, I know for sure, not all blind people do this, nor can do this. Not all blind people have a sense of aiming their heads at people who are talking to them, and I recall some being... critiqued for it at a blind school where I lived a few years, long, long ago.
I suppose the glasses could train you to aim well over time. It would be better than being yelled at and demeaned...

I agree that it would be amazing to be able to use smart glasses as an external camera with any app you want, and limitations in iOS and Android are indeed what prevents this. There is currently no way for software to provide a camera to the native camera APIs on either OS, and therefore no way to connect a wireless external camera to an app unless that app explicitly supports it. External USB cameras are supported on Android, and on iPads, but not on iPhones. However, I think apps must still explicitly support this, and this does not allow cameras to be used wirelessly.

However, I am currently planning a project that would run on a Raspberry Pi or other single board computer, and among other things, be able to send photos or video from smart glasses like the Seleste glasses to different AI models, as well as Android apps running in an Android container on top of Linux. I am planning for it to be controled by voice, and also to be able to control Android, as well as Windows and maybe other platforms, using AI and voice commands. It should support any Android, Windows, or Linux app, including Seeing AI, Aira, Be My Eyes, video calls in programs like Zoom and Google Meet, and hopefully even WaveOut, which uses the camera to help with providing navigation instructions. Doing this with iOS apps would be more dificult, but perhaps could be possible as well. I cannot promise any release date or timeframe since I am barely just starting this, but perhaps it will become the solution you are looking for.

Only problem is that it only works on Android because the power bus doesn’t supply enough power on iPhone. The other problem is that it requires a wire to be connected to your phone. It’s not cheap either, £600 introductory and £800 standard. People have also complained about the app. I know it’s not what you’re looking for really but it’s an option that’s worth mentioning as nobody had yet. Good luck to the comment above. You never know if it might turn into something thats marketable.

I had a review unit of the ARX last year. At that time at least, it only connected with their own app, so no different to Envision or Celeste. Unlike Envision, they did not have any integrations with Aira or other services either. Perhaps that has changed since though, I hope so. They did reach out to me a while back about a 2nd generation, but I haven’t heard any more as yet.
While I liked the form factor, I wasn’t impressed with the performance of the 1st generation last year.
Anyway, this is straying away from the original topic into general hardware talk, so I’ll stop my rambling here ☺️
Dave

I think they are trying to sell a package with the blind shell. I really don’t think its what we’re after here but thought I’d let you know about it.

When I first got an iPhone and discovered it had a camera, I was very skeptical. My brother is also blind and had been an iPhone user for at least a few years at the time, but I still wasn't convinced that I could use the camera or the photos app. Then I discovered Seeing AI, and was pretty impressed at what that app could do. I still wasn't taking pictures, but I began exploring Seeing AI. I eventually began using the app for some things. Then I discovered some of the other scanning apps and began using them for more things. Fast-forward to today. The iPhone 7 which I had was great, but I am now the proud owner of an iPhone 14. I have had somewhat better luck scanning things such as frozen meals. My aim is still hit-or-miss, but just last night I successfully read the contents of a Boston Market frozen meal which I pulled out of the freezer for supper. It happened to be the last frozen meal in my current supply, and a sighted neighbor was kind enough to tell me what it was. But Seeing AI did a great job last night, and so have these other apps. It's not perfect, but I for one am so happy that we have free and unlimited access to more information than we ever did. My point here is in the subject line, and hopefully nobody will take it the wrong way. I think an external camera attached to an iPhone would be awesome, but what we have is good enough for me at this time. Now if only I could start using ScripTalk to scan my meds, as the current supply has identical pill bottles! Perhaps future shipments will be like this too, but only time will tell.

My guess is the glasses that appear to talk to the phone are sending a wireless data package specfic to their app.

I completely agree though. I want to choose what app to use for each task. Facetime to talk with friends, get help, seeing ai when holding a box/trying to read an address, etc. It does seem to be the golden ticket.

The issue we have with all the options now is you're lock in. Meta, who suck, a startup who, with the best of intentions, may fail after a year. Gone is the time we buy hardware and choose how we use it. hardware is now simply an access point to a service for which we either pay or are data mined.

the Raspberry pi project is a good one, though has been done. The VOICE, seeing with sound, had a project with similar intent but, it kinda misses the point... We have all these apps, we just don't want them looking through our phone camera. I did have a play with a head mount for my iPhone 12, but, aside from looking like a twerp, it was very uncomfortable. I've got too much pride in my appearance to go out looking like a darlic too.

Siri control, and external camera will be the fix. How it can be done, who knows?

A quick search turned this up:

[Support external cameras in your iPadOS app(https://developer.apple.com/videos/play/wwdc2023/10106/)

I don't really understand such things, but even if it were through iPad, there are options. The iPad mini is at least easy to slip in a small backpack.

Have you explored phone head mounts? You end up looking like an utter tool, maybe even a darlic, but it could be an okay in house solution.

I had previously signed up for early access with vpn. But as of v4 for the glasses, Meta AI can provide descriptions in English without vpn in my country, and even recognizes points of interest without any problems. I believe it should be available worldwide very soon, even if you don't sign up for early access.

Which is your country?

VPN was going to be my next suggestion.

My country is Turkey. I managed to get early access by changing the vpn location to the US. But until a week ago, even if the vpn was turned on, the look and ask feature was not available. After updating my meta glasses, I can now get photo descriptions without using vpn.

That's interesting. Did you update using the VPN?

Anyone else had any luck and look and describe feature in any other countries?

After failing with the VPN I didn't try again, I did the updates through my local connection. I also got an email from meta about early access and was directed to fill out a form, maybe that had something to do with it.

Go-Pro is already a wearable gadget, to an extent. It will need a bit larger battery if it's to serve us entire day, clicking pictures and transmitting to Iphones.
I always love the ideas which aren't utilizing a dedicated disability specific hardware, rather achieving the same with usual general tech. It's more sustainable and diverse that way.

It’s frustrating because nothings really quite there yet. But by 2028, I would say we’ll have smart glasses be a part of our lives on a day-to-day basis.

Personally, I would really like AirPods with built-in cameras. There was an article several months ago stating that Apple has been in currently is working on them. I think it’s really just a matter of time. Right now, I would say we’re really trying to figure out how to utilize this artificial intelligence in a worthwhile way. iOS 18 could be pretty interesting. Realistically though, I think the Vision Pro glasses are going to be what we end up using. The second and third generation are going to be really interesting devices. They’re definitely gonna be smaller, better battery life, more advanced. I would say 2026 is probably when the second generation Vision Pro comes out and it’ll probably be around 2000 bucks. And then in 2028,the third generation Vision Pro glasses will probably come out and they will probably be something like $1000. There is a solid chance that that’s going to be an iPhone replacement. It’ll just be glasses that are cellular, Wi-Fi compatible, have cameras, and do all the things we do right now and more by that point.

I know it’s hard to wait for. But it’s going to come sooner than you think. It’ll change our lives because it’ll be a replacement for our eyes. The other day, I was going to Doctor. The GPS got me 99% of the way there. And I could not figure out which building was the actual address I was looking for. And then it occurred to me that I could use Be My Eyes. And the AI literally told me which house had the address number I was looking for. It made something that could be so frustrating incredibly easy. With a built-in camera it would’ve been even easier. I’m really looking forward to it. I imagine walking through New York City and being able to get feedback about which stores I’m passing or what street vendors I’m passing or what street I’m crossing or the pretty women that are smiling at me, and I wouldn’t known before.
Just be patient. It’s going to happen.

I'd not hold your breath over the AirPods with cameras. They are being researched, that's it. Apple tries many devices that they don't take to market.

The issue with AP with cameras is one of geometry, a camera has to be able to see past the cheeks of the wearer, hair, beard etc, and display what is in front of the wearer. This won't be possible without the camera module protruding a significant way making the device bulky, most likely, looking rather odd. Added to that, the extra weight will make them even more uncomfortable for prolonged usage.

the device we need needs to be worn all day, in comfort, without any possibility of the field of view being occluded.

Wearable camera devices are certainly on the way, just don't hold out for ones that are in AirPods.

In the naming things department. You’re a natural. Air Frames, AIpod and Apple Core are pure genius and you should charge them a fortune to use your ideas.

Hello,

I found this info about Meta AI and Llama 3 from the last days about languages and countries:
- "included four times more code than in the Llama 2 training dataset and that 5% of that set has non-English data (in ~30 languages) to improve performance on languages other than English.
- Meta says that it’s currently training Llama 3 models over 400 billion parameters in size — models with the ability to “converse in multiple languages,” take more data in and understand images and other modalities as well as text, which would bring the Llama 3 series in line with open releases like Hugging Face’s
https://techcrunch.com/2024/04/18/meta-releases-llama-3-claims-its-among-the-best-open-models-available/

And from the Meta website:
- We first announced Meta AI at last year’s Connect, and now, more people around the world can interact with it in more ways than ever before.
We’re rolling out Meta AI in English in more than a dozen countries outside of the US. Now, people will have access to Meta AI in Australia, Canada, Ghana, Jamaica, Malawi, New Zealand, Nigeria, Pakistan, Singapore, South Africa, Uganda, Zambia and Zimbabwe — and we’re just getting started.
https://about.fb.com/news/2024/04/meta-ai-assistant-built-with-llama-3/#:~:text=Now%2C%20people%20will%20have%20access,more%20fun%20than%20ever%20before.

Good spot. Interesting there is no mention of the UK.

We need glasses that actually look like glasses and sort of fashionable.
I had the Envision glasses for a few minutes and let's be honest, they look like a pile of crap that wanted to be a pair of glasses but failed while trying. That hole things screams look, I am assistive tech and that person wearing it is some ideot who doesn't know anything substantial.
Let's forget design problems for a minute, my ideal pair of glasses would something like this:
a good looking, sturdy pair of sunglasses, either normal or wrap around glasses, both would be fine.
The cameras would be integrated into the lenses, so they don't sit on the side like in the crapshoot I mensioned above, but they are right in front of your eyes. both lenses have an ultra whide and zoom lense integrated.
The glasses have a touchpad that is magnetic and can be attached either to the glasses themselves or another surface, like your jacket, shirt, wristband or where ever you want.
The glasses would choose automaticaly which lense to use for which operation, if you want to take a picture of the current scenery ahead to have it described for you it would use the ultrawhide, if you want to scan a piece of paper to red some text, it would go with the zoom one. You always could change though which lense is used, in situations where you are out and about and just want to know what is in front of you,you could switch from the ultrawhide to the zoom one.
There, this is how I would imagine a product like this.
I really hope that apples vision pro develops further and we might see something like this in a cupple of years.

Back in the 80s, and where I lived, there was a bit of a backlash going on by blind people with wearing darkened glasses. Mostly because many of the famous blind people wore shades so black, you could wear them for arc welding or eclipse viewing. And I've got to admit, just the sunglasses part, however stylish, causes this faint sense of intimidation in the back of my mind, creeping up from that era of my life, and those who preached against wearing shades when your blind.
Oh, but it could be any number of things, like the "correct" cane length; dog or no dog; even folding canes or rigid with a crook; whether dating other blind people etc should be frowned upon...
So yes, Lottie, to someone, somewhere,, blind or otherwise, having a guide dog probably does say exactly that, along with a thousand other things about us.

Remember when I was sighted saw a movie with Stevie Wonder, he was very young, he had this big dark glasses that look strange. It was one of those old movies with the beautiful Annette Funicello.

For £3000 and it is huge! But it is a 'smart CoPilot for the blind and visually impaired' so it must be good? Right?

I don't know about you, but I know it when I see it. This isn't it.