Apple Previews Upcoming Accessibility Features and Enhancements, Including New VoiceOver Voices, Braille Upgrades, and Magnifier Reader Mode

By AppleVis, 15 May, 2024

Member of the AppleVis Editorial Team

Apple is once again celebrating Global Accessibility Awareness Day (GAAD) by offering [a preview of accessibility features coming later this year.

For VoiceOver users, improvements include new voices, a flexible Voice Rotor, custom volume control, and the ability to customize keyboard shortcuts on Mac.

Braille users will benefit from a new way to start and stay in Braille Screen Input for faster control and text editing, Japanese language availability, support for multi-line braille with Dot Pad, and the option to choose different input and output tables.

Low vision accessibility sees the introduction of Reader Mode in Magnifier and the option to launch Detection Mode with the Action button; Hover Typing shows larger text when typing in a text field, and in a user’s preferred font and color; and features like Reduce Transparency, Smart Invert, and Dim Flashing Lights on visionOS.

Also coming to visionOS is Live Captions, which will transcribe audio in real-time during spoken conversations and in audio from apps, benefiting deaf/hard of hearing users. Apple Vision Pro will gain support for additional Made for iPhone hearing devices and cochlear hearing processors.

Among the other notable new features is Eye Tracking for iPad and iPhone, which uses the front camera and on-device machine learning to allow navigation through the elements of an app and use Dwell Control to activate each element, accessing additional functions such as physical buttons, swipes, and other gestures solely through eye movements. This will provide a built-in control method for users with physical disabilities. It will work across iPadOS and iOS apps, and doesn’t require additional hardware or accessories.

Another addition is Music Haptics, enabling those who are deaf or hard of hearing to experience songs through taps, textures, and refined vibrations to the audio of the music from their iPhone's Taptic Engine. Developers will also gain access to an API for haptic music integration.

While outlining these new features, Apple has not provided much detail on exactly how they will be implemented or the specific user experience for accessing and using them. More information may be shared at next month's Apple Worldwide Developers Conference (WWDC). If not, we'll likely have to wait until the beta releases become available later this summer to get a closer look at how these accessibility advancements will work in practice.

Additional Enhancements

  • Accessibility features coming to CarPlay include Voice Control, Color Filters, and Sound Recognition.
  • With Vocal Shortcuts, iPhone and iPad users can assign custom utterances that Siri can understand to launch shortcuts and complete complex tasks.
  • Voice Control will offer support for custom vocabularies and complex words.
  • For users at risk of losing their ability to speak, Personal Voice will be available in Mandarin Chinese. Users who have difficulty pronouncing or reading full sentences will be able to create a Personal Voice using shortened phrases.
  • For users who are nonspeaking, Live Speech will include categories and simultaneous compatibility with Live Captions.
  • For users with physical disabilities, Virtual Trackpad for AssistiveTouch allows users to control their device using a small region of the screen as a resizable trackpad.
  • Switch Control will include the option to use the cameras in iPhone and iPad to recognize finger-tap gestures as switches.

More Apple Celebrations of Global Accessibility Awareness Day:

This week, Apple is introducing new features, curated collections, and more in celebration of Global Accessibility Awareness Day:

  • Throughout the month of May, select Apple Store locations will host free sessions to help customers explore and discover accessibility features built into the products they love. Apple Piazza Liberty in Milan will feature the talent behind “Assume that I can,” the viral campaign for World Down Syndrome Day. And available year-round at Apple Store locations globally, Today at Apple group reservations are a place where friends, families, schools, and community groups can learn about accessibility features together.
  • Shortcuts adds Calming Sounds, which plays ambient soundscapes to minimize distractions, helping users focus or rest.
  • Visit the App Store to discover incredible apps and games that promote access and inclusion for all, including the accessible App Store Award-winning game Unpacking, apps as tools for augmentative and alternative communication (AAC), and more.
  • The Apple TV app will honor trailblazing creators, performers, and activists who passionately share the experiences of people with disabilities. This year’s theme is Remaking the World, and each story invites viewers to envision a reality where everyone is empowered to add their voice to the greater human story.
  • Apple Books will spotlight lived experiences of disability through curated collections of first-person narratives by disabled writers in ebook and audiobook formats.
  • Apple Fitness+ workouts, meditations, and trainer tips welcome users who are deaf or hard of hearing with American Sign Language, and Time to Walk now includes transcripts in the Apple Podcasts app. Fitness+ workouts always include Audio Hints to support users who are blind or have low vision, as well as modifiers so that users of all levels can participate.
  • Users can visit Apple Support to learn how their Apple devices can be customized using built-in accessibility features. From adapting the gestures to customizing how information is presented on a device’s screen, the Apple Accessibility playlist will help users learn how to personalize Apple Vision Pro, iPhone, iPad, Apple Watch, and Mac to work best for them.

Apple has not provided any specific release dates for the upcoming features; however, they are anticipated to be included in the next versions of their operating systems, such as iOS 18, iPadOS 18, macOS 15, watchOS 11, and tvOS 18, which are expected to be launched this fall.

We'd love to hear your thoughts on the upcoming accessibility features from Apple! Feel free to share your opinions with us by leaving a comment below.

Options

Comments

By Ollie on Saturday, May 18, 2024 - 06:41

The BSI, for me, is the most interesting of these. After playing with Android I'd say that IOS has fallen behind with the ease, convenience and reliability of BSI. Hopefully this will go some way to solving such issues.

By Dennis Long on Saturday, May 18, 2024 - 06:41

I'm excited about iOS 18 and look forward to trying it when the betas are available.

By Holger Fiallo on Saturday, May 18, 2024 - 06:41

I am curious how many people use some of the features Apple releases every year. Applevis should do one to see how many of us do so. Have 13 pro but never used door dictation or magnifying to get VO to tell us what it is.

By Bruce Harrell on Saturday, May 18, 2024 - 06:41

Dear Apple,
No matter how many uncorrected VoiceOver bugs pile up against the blind who use IOS and MacOS, I am most sincerely glad to hear you are addressing the needs to the more severely disabled on Earth. Some of your accessibility improvements are and will be an enormous boon to those who use them. My hat is off to you. Bravo!

By Gokul on Saturday, May 18, 2024 - 06:41

Or does anyone else think this compares poorly to all the other things which have been revolutionising accessibility out there recently?

By Enes Deniz on Saturday, May 18, 2024 - 06:41

The annual survey already covers everything. I am talking about the accessibility report thing. Don't recall what exactly it's called.

By Jonathan Candler on Saturday, May 18, 2024 - 06:41

I wonder how many voiceover bugs we'll see this time both on mac and IOS? I wonder why apple can't just focus on fixing bugs instead on adding new features that will more than likely break the existing codebase? Sure bugs do get fixed but there are still a lot that hasn't been addressed for quite a while now and there gunna come a time when Voiceover is gunna need a complete over hall if it doesn't need it already! I'm gunna go off a bet and say that the existing code is a mess and has been for quite some time given the nature of bugs, and features combined over time. They keep adding things to said code instead of cleaning it up so that newer features can be added which is not, how you do things. It's also a great way of introducing bugs that shouldn't be there in the first place. I'd like to be wrong on all this though and if I ain't, I can safely say, I told ya so. However, some of these features, particularly one about BSI peaked my attention as that's my most way of typing on my phone. You saying I can have BSI as my default typing method when I go into the keyboard? Finally! That needed to be in there for a long time and they just now getting into adding this? Guess better late than never I suppose.

By Dennis Long on Saturday, May 18, 2024 - 06:41

Bug fixes will come. They wouldn't be announced today. You have two options on June 10th either sign up for the developer beta or when the public beta is available use it and give feedback.

By Brian Giles on Saturday, May 18, 2024 - 06:41

For me, the BSI stuff and the action button for detection mode are what I'm looking forward to. I upgraded to a pro expecting something new that uses the lidar sensor, but a super quick way to pull up detection mode for the stuff that is already there will be useful. Yes, you can four finger triple tap, but the action button will be much easier. Now if I could figure out how to actually make point and speak work. lol

New voices, meh. But I know this forum will have a field day with that since people are so particular about the voices they like. I do wonder what they mean by "flexible voice rotor" though.

By TheBllindGuy07 on Saturday, May 18, 2024 - 06:41

Are the most underated feature and what I am the most excited for. The current keyboard layout is nothing than poor especially for those used to windows and it will be interesting to see how far we can customize them. Nothing about screen recognition on mac though which is sad but I guess it will slip through any of the apple ai thing anyway. This reminds me that, even if most of it is arguably marketing, we must remember that most of the blind users are vocal on this website but we are not the only category in the disabled people and what apple does, even on paper, is litterally more than every big tech out there currently. So good job apple. Fix voiceover at least a little bit, fix the calculator on macos and I'd be happy.
Most of my angryness to mac is because even the most trivial tasks seem near impossible to do such as text editing / formatting in Pages or for code but since I've found vscode being very usable... I am treuly able to leverage my mac now! Bad accessibility is still better than no accessibility at all, try anything other than gnome or mate on any of the linux distro with gui. Looking forward to see all of this.

By Jonathan Candler on Saturday, May 18, 2024 - 06:41

I'm well aware that bug fixes don't get announced in a release like this. That's not the point. I would love to join the dev beta rather than public beta however, as I only have one phone and nothing else, that may not happen this time around and I use my phone a lot for work so can't have that risk of things breaking when I don't want them to. Public beta is fine enough and I had no issues with that but I'd like to take a step further though. One of these days would love to have a device, simply for testing.

By PaulMartz on Saturday, May 18, 2024 - 06:41

Thanks David and AppleVis team.

I've seen new features come and go. Very rarely do they significantly impact my productivity. And many new features have fallen into disrepair over the years, or never lived up to their initial promise, handoff being one memorable example. Consequently, it's hard for me to get too excited.

I think most Apple products would be at a very satisfactory level of accessibility if bugs and usability issues were addressed. A bug example would be Safari Not Responding. Usability issues include the VoiceOver contortions required for sidecar, iPadOS split view, or pretty much anything in Mac Garageband. This could easily be marketed if it were spun correctly. Early Apple ads that painted the Macintosh as easy to use were quite successful. I can imagine an Apple marketing campaign that touts how easy it is for marginalized demographics to use Apple products. Of course, they'd need to actually address the issues first.

The ability to customize keyboard shortcuts on Mac is intriguing. We currently have a confusing palette of keyboard shortcut options, each with their own limitations and esoteric configuration screens. If Apple has opted to clean up the present cornucopia and introduce a flexible and coherent redesign that follows a tools-not-rules philosophy, that would be a welcome change.

I'm unclear about the vocal shortcuts feature that will allow SIRI to complete complex tasks. Many leaks indicate SIRI will get an AI redesign so that it will understand general commands. If so, configuring a specific vocal shortcut seems moot. I'm unsure why this separate feature would be necessary.

Will old features be updated to take advantage of new features? For example, the iOS Measure app is pretty challenging for blind people to use, but I can imagine it being much easier if it were enhanced with the new AI capabilities that we keep hearing about. And I'm still waiting for Raise To Wake to become smart enough to not wake when it's being jostled in the pouch of my hoodie. If these old features will not be redesigned, that's a missed opportunity to improve products overall.

By Ollie on Saturday, May 18, 2024 - 06:41

I hope we're bombarded with feel good videos of blind people doing incredible things, previously unimagined... Like surfing the web without launching ones mac through the nearest wall.

PS... Spell check is still broken in safari when typing into text boxes such as this one. Absolute jokers.

Just imagine, for the international day of disability Apple announces they've got off their high horses and fixed just one or two of the mirad broken aspects of voiceover on mac.

*Smash zoom into my smiling face*

Now that would be an ad.

By Lielle ben simon on Saturday, May 18, 2024 - 06:41

I excited because I will be choose reading and writing Braille tables like it's possible to do with Jaws.
I can read and write in Hebrew but if I moove Braille table to write in English, i can't read in Hebrew.

By Ekaj on Saturday, May 18, 2024 - 06:41

I'm just seeing this now and have to comment. As always, it seems that the good folks out in Cupertino are at it again. This is fantastic. I'm curious as to whether or not this new reader mode for the magnifier will further help in aiming my phone correctly. In addition, I might be getting one or 2 hearing aids and am looking forward to trying out Apple's other hearing accommodations . I've tried out the background sounds on both my devices. It was recently discovered that I have Eustachian-tube dysfunction, and I'm getting another hearing test this weekend.

By Holger Fiallo on Saturday, May 18, 2024 - 06:41

No. That one does not do a good job.

By Kushal Solanki on Saturday, May 18, 2024 - 06:41

The new way to start BSI and stay in BSI starts very interesting.
Can't wait to beta test.

By JC on Saturday, May 18, 2024 - 06:41

those features sound interesting. I cannot wait to test.

By Brooke on Saturday, May 18, 2024 - 06:41

I'm intrigued by the changes mentioned for BSI. I've started exploring Android again, and their implementation of BSI is ahead of Apple's.

By Levi Gobin on Saturday, May 18, 2024 - 06:41

I am really excited to be able to change the default keyboard shortcuts on Mac, which you have been able to do since iOS 13.
I feel like every version of iOS, at least for the last couple of years, they’ve been adding new voices. in iOS 16, it was eloquence and the novelty voices. In 17, it was the US natural Siri voices that are a lot smaller in size.
What voices do you think they will be adding this year? More Siri voices? Maybe some new unseen and unheard of vocalizer voices? Older retro speech synthesizers? What are your thoughts?
Also, out of curiosity, if you could have any voiceover voice in the world, what would you choose and why?

By Zachary on Saturday, May 18, 2024 - 06:41

I'm pretty sure these are new custom voices by Apple, I don't believe Cerence has anything new for Vocalizer at this point. I for one would welcome Apple getting back into the TTS game, Alex was the last big project they really worked on besides the stuff for Siri. If it's Vocalizer thats great as well, but I really hope they move away from that and start working on their own tech instead to catch up with Microsoft and Google.

By Dennis Long on Saturday, May 18, 2024 - 06:41

It is still the best voice out there. I'm glad apple added it.

By ming on Saturday, May 18, 2024 - 06:41

well,
I hope theyc an fix the current voice over issue first.
like some of the high quality voices is not really high quality!
and the siri voices will disappear when we charge our IPhone

By Brad on Saturday, May 18, 2024 - 06:41

They'll probably be the chat gpt voices.

By Ollie on Saturday, May 18, 2024 - 06:41

"We went ahead and made it possible for you to hear "Safari Not Responding" in a range of new and highly realistic human sounding voices."

By SSWFTW on Saturday, May 18, 2024 - 06:41

I would love to see something like this quality but from what I understand speech with this inflection and human like Quality is generated in away that wouldn't be great for on the fly kind of stuff

By Enes Deniz on Saturday, May 18, 2024 - 06:41

If the new, smaller Siri voices can synthesize speech on the go, why can the OpenAI voices not do the same on-device? It is unfair that the smaller Siri voices are only available for English and possibly not even all dialects of it, but they're not significantly less responsive than the Vocalizer voices or the large Siri voices.

By Ollie on Saturday, May 18, 2024 - 06:41

Because, I think, the audio is compiled, like an MP3 or similar is created to play back the voice. A screen reader is instantly reactive but can't add in all the AI inflections as it has no understanding of the text it is reading. I think they are two different things. I'm not sure if I'd want too much inflection when I'm just typing, or navigating my mac/iPhone anyway. When it comes to documents, however, I'm hoping they will allow for a more AI driven way of reading long form.

If you try Eleven Labs, and everyone should, it's bonkers good, there is a period of compilation time. It will get faster I'm sure, but I think the navigation voice needs to be responsive, the reading voice is the one we want upgrading to sounding like David Attenborough.

By Enes Deniz on Saturday, May 18, 2024 - 06:41

The new AI-powered Siri voices will also have to synthesize speech "instantly", and the US English voices already do that and are quite responsive indeed. This is what I'm trying to say. The new Siri voices are different from the large ones in the sense that they don't directly make use of hundreds of megabytes of data created out of recording snippets and instead synthesize speech in real time.

By Ekaj on Saturday, May 18, 2024 - 06:41

I'm always intrigued to try out new voices, and this year is no exception. I'm currently using Alex on my Mac, and I switched to Matilda just a bit ago on my iPhone. I believe I've heard several of the voices from Eleven Labs, and they sound fabulous. Drew sounds like David Letterman to me. Having said all that, I'm happy with the voices that are currently available and it's cool that they can also serve as Siri's voice. The Per-Voice settings which were added are great, and I hope Apple expands them because it seems there are a few voices that don't have this. In addition, TruVoice would be neat. I used those under Windows, and one of the females has my mother's first name. I'm not saying which she is, but the fact that she and my mother share the same name is very sweet.

By Holy Diver on Saturday, May 18, 2024 - 06:41

I'm surprised how much that extra step to do BSI on iOS every time still irks me. It shouldn't bug me so much to just do one rotor gesture and get to my BSI on the top of that silly thing but it really, really does. meanwhile on android I just double tap the edit field and it's right there. I know it realistically doesn't add that much time on the I devices but that matters to me way more than seems logical.

By TheBllindGuy07 on Saturday, May 18, 2024 - 06:41

in his latest video on youtube. This gives us more visibility than ever in public, plus there are a handful of voiceover users in the comments there, I guess most of them knows applevis as well.

By TheBllindGuy07 on Saturday, May 18, 2024 - 06:41

I'll be happier than ever. Unlikely though but I can hope!

By Enes Deniz on Saturday, May 18, 2024 - 06:41

Perhaps it will let us add some voice-specific parameters to the rotor actions?