Introducing EyeSense, an object and facial features recognition app for the blind

By Not needed, 28 July, 2016

Forum
iOS and iPadOS

EyeSense is an Artificial Intelligence application that uses deep learning-based computer vision to recognize objects and special facial features, like smiles. The application can be used to help blind and visually impaired to interact with their environments.

Objects Recognition:
EyeSense can easily identify your personal objects. All you need to do is point your device camera towards your surroundings.

Smile and Wink Detection:
To improve human connection, EyeSense reads aloud facial expressions, like smiles and winks, registered on a person’s face.

Voiceover:
Simply tap the device screen to hear EyeSense identify previously learnt object.

Adding Text Description:
You can help EyeSense learn any of your objects by adding a text descriprion. Just double tap the screen and the text will appear again.

Teaching EyeSense:
You can add your personal objects in two simple ways;

1. Open the application, press a Learn button, type a description of your object and add a range of positive* and negative** number of image samples (recommended range of acquired images is from 100 to 200). You should move then your camera around a new object to see it from different camera angles and positions.

2. Upload directly a photo album of positive and negative image samples from your photo library.
You can always add more training images to previously learnt objects to enhance the application performance.

*Positive images are images that contain the object you want EyeSee to recognize.
**Negative images are images that do not include the object of learning. They should be captured in the same backgrounds where *positive images were taken but without the object of interest being present in the scene.

https://itunes.apple.com/us/app/eyesense/id1098335532?mt=8&ign-mpt=uo%3D8

Facebook Community Group > https://www.facebook.com/groups/681966438622119/?ref=bookmarks

Options

Comments

By Marconius on Tuesday, July 26, 2016 - 03:26

Why would you make an application for the blind that relies so heavily on photography skills? As for facial recognition, after having completed a weeklong study for a similar technology, I never want to be in a situation where I'm using my phone to read people's faces, whether in private or out and about in public. It always made me feel like a creep. Unless it can describe the photos after the fact, it's just not a feature and personally interested in.

This app sounds like it needs quite a lot of set up in order to be functional, and is asking for a ton of data that would be very difficult to provide when you can't see what you are inputting.

By Siobhan on Tuesday, July 26, 2016 - 03:26

We all know we have cameras, I'm willing to bet though probably 90% of blind people don't even use them, for reasons the first commenter has submitted already. I do agree i would not use it out and about, I'm just not afraid to try new things, so yes this app does seem a little hard to navigate anduse, I'll jump in face first though. Like it or not, We're in a sighted world, let's start interacting with it, like people who already aren't gawked at because they don't use a phone like John Public does.

By Jake on Tuesday, July 26, 2016 - 03:26

If you don't want to be gawked at, I'd suggest not using something like this. You do not want to be taking pictures of things in your environment or worse, unsuspecting people. You might get worse things than a gawking if you do!

By splyt on Tuesday, July 26, 2016 - 03:26

Way way over criticism.

Agreed that blind people are not the best audience to teach the AI to recognize things but they are a great audience to use such softwares to help them in their daily lives.

What is missing here is strategy: do engage the sighted comunity to teach the AI by spending five minuts of their busy time per day pointing at and tapping the maximum of objects they can and collect this into a database of information to be used latter on by the AI to efectively assist blind people.

Those who might have doubbts about the potential of engaging the sighted comunity just take a quick look at what by my eyes did and the doubts should no mor exist after this analysis.

The number of good devs who want to help blind people by implementing every possible types of apps is incredibly huge and I am for one very grateful to all of them but the number of them who won't talk to blind people before implementing their ideas is also big, which ends up making so many efort almost null.

Now folks instead of criticising like you've just done try to give back ideas, like I did. More useful ..

By Monica Rose on Tuesday, July 26, 2016 - 03:26

I try to keep an open mind with these kinds of apps. I'm not good with a camera at all, and these never seem to work for me. I wouldn't know what range to put in these positive and negative fields. However, I'd like to learn. It sounds a lot like the Aipoly app. I've not been successful with that app either, but I can see where this kind of app might be useful. If I'm somewhere other than home, perhaps it might help me be aware of what's around me. Thank you so much for trying to make an app beneficial to blind and visually -impaired people. I'll download it and try to use it. I'll more than likely have many questions.

By Wendy on Tuesday, July 26, 2016 - 03:26

The EyeSense app changes my life. I don't have to ask some one witch object is mine. For example there are 3 sunglasses on the table and I want to leave, now I just take for now my iPad and with the cam I move around the table. And yes the app says Wendy's sunglasses.
Why I learn the app to recognize only my objects. With other app I could not do that and even do they recognized "an" object it stil didn't say if it was mine object and still had to ask some one else to help me.
With the EyeSense app I no longer have to do that and you don't have to be a pro to work with the cam. A kid can do the laundry, I can life my live now more independent.

By Imaginingstuff on Tuesday, July 26, 2016 - 03:26

I think this ap sounds pretty cool! The idea to have sighted people fill the database with recognizable pics it excellent. They could put in different types of flowers and trees and birds and dogs and sure smiles too! I would love to walk down the street and take a picture of a flower along the way and know what its name is.

The best ever!
A

By Kristen on Tuesday, July 26, 2016 - 03:26

This does sound interesting - the teaching of the AI does sound daunting with the amount of photos that need to be taken, but could be useful with sighted help.

Is this not available in Canada? I tried searching for "EyeSense" and didn't get any results.

By Ken Downey on Tuesday, July 26, 2016 - 03:26

Other than teachability how does the app differ from, say, tap tap see?

In tap tap see it calls objects in general. Now with the learning button you can personalize your object in EyeSense app.
Like; 3 sets of keys on the table , tap tap see says keys, keys, keys.
EyeSense says after learning ; keys, keys, Wendy's keys. So you don't have to ask someone else witch set of keys is yours.
With EyeSense you only have to let your cam go over the table and it will recognize your own object you learned EyeSense.

By chris R on Tuesday, July 26, 2016 - 03:26

Hi.
It seems this app is only for the iPad, at least that's what I have found after following the link in the post above. I'm just interested why there isn't an Iphone version. I would like to try the app but I don't have an Ipad. I would have thought it would also be easier to take photos with the smaller Iphone rather than the Ipad.
Chris.

By Not needed on Tuesday, July 26, 2016 - 03:26

In reply to by chris R

Thank you for your comment, a new version for iPhone will be available soon with extra features

By Wendy on Tuesday, July 26, 2016 - 03:26

In reply to by chris R

The iPhone is coming but we also need funding to realize the app. Only the sun comes up for free. They know the issues they are working on it, also for security reasons. For now you can get to know the app so you get know with it. Update will follow.

By Caroline on Tuesday, July 26, 2016 - 03:26

I've just downloaded this app on to my I pad and there doesn't appear to be any instructions this means I'll have to keep referring back to this post its seems too much of a faff for me so i won't be using it

By Santiago on Friday, August 26, 2016 - 03:26

I like the idea of using artificial intelligence for image identification, however, I'd personally like it being used differently. For instance, if this functionality was built-in to the iOS camera app, a user could easily tell how many people are in the frame, and their expressions while taking a photo. The user could also know whether other objects are in the frame as well before taking a photo, such as animals, cars, trees, etc. I've read a couple of articles which are publicly available online that claim iOS 10 will have object recognition functionality for the camera roll, so in theory, doing something like this is technically possible. I really wouldn't use this to always know details of our surroundings. Pulling out the phone and moving the camera around, especially towards people without taking a photo is a bit odd, and may be uncomfortable for some.

By Not needed on Friday, August 26, 2016 - 03:26

In reply to by Santiago

Thank you for your thought and actually yes, one good feature about EyeSense that you don't need to take any photos, you just open the app and it will automatically take the video feed from the device camera and tell you what the camera sees.