Hi all,
I am a blind master's graduate in Computer Science, specializing in Machine Learning and Computer Vision. My experience is primarily research-based using ML tools in Python, rather than software development using Swift for Apple platforms.
One of the things I miss as a blind person is the ability to take photos properly. Given what Google offers on Pixel phones with Guided Frame, I looked for a similar experience on iOS, but nothing came out of my research. Therefore, I started learning Swift and iOS programming, although I'll admit that I am still a beginner in this domain.
In parallel, I wanted to apply my knowledge in Machine Learning to help solve this problem. This involved a lot of trial and error, and a few months of trying different approaches. The main limitation I faced was a lack of resources—specifically, the hardware required to train a proprietary model, and a high-quality dataset of daily subjects representative of what a blind individual might encounter.
Given this gap, and the recent advancements in iOS with regards to Machine Learning, I used AI-assisted development to create an initial "proof of concept" called SceneWhisper. It exclusively uses Apple APIs to check the feasibility of such an approach.
What I am looking for:
- Testers: People who are tolerant of a process where the concept is still in its infancy. Bugs are likely, and there is no guarantee the app will become a final product.
- Connections: People willing to connect me with entities who might help bridge the resource gap.
- Feedback: Anyone interested in reaching out to talk about the POC, given the constraints mentioned.
I am also a solo developer pursuing a PhD, so please bear with me!
About the App
The app today can help take photos of subjects and people. Currently, only speech guidance is provided, with the possibility of creating parallel or complementary haptic and auditory experiences. Additional modes for scenes and documents are being considered.
The goal at this stage is to check whether the app can provide reliable feedback to take a shot, given the limitations of the available APIs, while keeping battery consumption within a normal range.
If everything goes as expected, the app will be free of cost, as I don't believe we need to pay for our rights, and it may be open sourced eventually.
Anyone interested in getting in touch is warmly welcome to drop me a line at mohammad.xciii@gmail.com. Alternatively, a private message using the forum is more than suffice as well.