Intro
So, you're a developer of Apple software. You've followed Apple's guidelines for making your iOS app accessible , or making your OSX app accessible to Voiceover users. Your controls are labeled, you have hints set up, you have things grouped into containers, and you've made sure everything plays nice when Voiceover is running. First of all, I applaud your efforts, as do all your blind and visually impaired users.
But now it's time to take things a step further. You've heard something about a "magic tap" gesture that only Voiceover users can activate, some nifty thing that lets them perform actions without needing to find exactly where a button is on the screen, like hanging up a phone call without having to locate the "End" button. Or maybe you have an area in your app that needs to take full control of all touch events, but Voiceover is interfering, but you know there's a way to get around this because Voiceover users can play instruments in Garageband just fine. Maybe you need Voiceover to speak a string as an alternative to a visual notification. Well, all this and more await you below.
Notes
First, this is a work in progress. Not every accessibility trick is here yet, and I hope to keep growing this as time goes on and APIs change with new iOS and OS10 releases.
Second, much of this work comes from Josh, Cara, Barry, and others on the MV-Dev mailing list. Thanks for all your help, everyone!
Third, this is for both iOS and OS10. Each section will state which platform it applies to, or if it applies to both. Where I know it, I will also include the minimum version that offers the support.
Fourth, all this is Objective-C, not Swift. Translating what you need should be easy, though.
Okay, let's get started!
Is Voiceover Running? (iOS)
Most of the time, your app does not need to know if any accessibility features are enabled. It should be labeled, tagged, and properly set up no matter what. However, there are situations in which it would be good to know if Voiceover is on. For example:
- You have a bar code reader, and you want to speak the identified item as soon as you know it, instead of making the user find the UI control that holds the name.
- You have a game, and want to alert the user when it is their turn. This is easy to do visually, but if the user cannot see the screen, the best way may be to have Voiceover speak the alert instead.
- A long job finishes, but your app's flow does not lend itself to popping up alerts, so you have a visual notification. You want to speak the information if Voiceover is on, so blind users are aware as well.
- You get the idea, I hope.
The Magic
To detect if Voiceover is on, just use this statement:
if (UIAccessibilityIsVoiceOverRunning()){
//do stuff with Voiceover
}
Focus on a Specific UI Element (OS X)
If your app needs to jump the user to a particular location, it's pretty easy to manage. Please be aware, however, that this can be quite confusing, especially if other parts of your UI change at the same time. Situations where forcing the user to a specific area can be helpful might be:
- You have a preferences window with several buttons in a toolbar, each button set to load a different screen. When the user activates one of these buttons, you could move focus to the first control of the newly loaded screen, so the user does not have to stop interacting with the toolbar and move to the control manually.
- Your app lets the user handle articles, such as RSS or a read later service. When an article is chosen from a list, it opens next to the list. Instead of leaving the user on the list, move focus to where the article is displayed. Be sure that you don't do this when a new item is focused on in your list, though, as that would cause the user's focus to jump every time they arrowed down to read the next article's title!
The Magic
In the view controller for the view holding the control you want, call this:
[[view window] makeFirstResponder:myControl];
This causes VoiceOver to move to the "myControl" element of your view and speak the control's title. Again, though, use this carefully, and test it with Voiceover (or better yet, an experienced Voiceover user). As in the third example above, you don't want focus to jump out of a table just because the user arrowed to the next item, for instance. Used correctly, though, this can be a real time-saver and make your app feel very polished and efficient.
Setting Focus to a Particular UI Element (OS X)
The way to do what the previous tip did, but on OS X, is easy enough. As before, though, be careful when forcing focus to move around. You may wish to do this when:
- The user goes back a page in your app, and you want to set focus to where they were when they last viewed the new view
- Opening a new view in your app presents some instructions, and you want VoiceOver to speak those instructions when the view loads
- You update your view, and want to put the user's focus on some new control
The Magic
UIAccessibilityPostNotification(UIAccessibilityScreenChangedNotification, viewToFocusOn);
Magic Tap (iOS)
When Voiceover is on, users can double tap with two fingers to perform actions. On iPhones, for instance, this will answer an incoming call, or end one already in progress; on other devices, or if no phone call is in progress, it will play or pause the active media playback app. The cool part is that your app can take advantage of this gesture. For example:
- Facebook uses it in their iOS app to let Voiceover users quickly access options for the currently focused wall post
- a self-voicing GPS app, Blindsquare, uses it to mute or un-mute its built-in speech
- a book-reading app, Voice Dream Reader, uses it to start or pause reading
- Prizmo, a scanning/OCR app, uses it to snap a picture of the focused paper
The Magic
If you would like to use the two-finger double-tap gesture (also called the Magic Tap) in your app, you can implement the following method in the view controller whose view you would like this gesture to be added to:
-(BOOL) accessibilityPerformMagicTap {
// Perform custom actions here
return YES;
}
NOTE: if you would like to have this gesture work from anywhere within your app, then you will want to add the above method to your app delegate. Also, when adding the magic tap gesture to a view, make sure your view does not include the accessibility trait UIAccessibilityTraitAllowsDirectInteraction.
The performMagicTap method returns YES when your action is successful and NO if it is not. As in:
-(BOOL) accessibilityPerformMagicTap {
if('Custom Action Succeeds')
return YES;
else
return NO;
}
The boolean returned signifies whether or not the touch has been captured, so it is possible to have several of these methods within your app which selectively let the magic tap gesture perform different actions within different contexts of your app if desired.
Let's assume that you have the performMagicTap method implemented in your app delegate and the below method implemented in a view controller in your app.
-(BOOL) accessibilityPerformMagicTap {
// Perform custom actions here
return NO;
}
IN this case, the custom action will be performed but the gesture will not register as being captured, and will then be received by the performMagicTap method in your delegate and be acted upon there. IN this way, if you choose, you may deal with the two-finger double-tap gesture in a more flexible and contextual way.
Direct Interaction (iOS, as of iOS5)
Sometimes, you need Voiceover to get out of the way, so your view can interpret touch events directly. Virtual instruments, exploring a map by touch, interpreting touches and gestures in a game, and so on. While you could check if Voiceover is running and pop up a warning that the user needs to disable Voiceover, that is hardly an ideal solution. A much better way is to simply tell Voiceover that, for this particular view, your app is in charge of touch interception, not Voiceover.
The magic
To tell Voiceover to ignore touch events for a certain view in your app, simply set the proper trait:
[mySubView setAccessibilityTraits: UIAccessibilityTraitAllowsDirectInteraction];
That's all it takes. Now, when the user is navigating your app with Voiceover, all will be normal. As soon as the user touches the view for which this trait is set, however, the view will start receiving touch events and Voiceover will do nothing. That is not to say Voiceover is disabled, however, so be sure to speak text, send accessibility notifications, and overall continue to support accessibility even though you are temporarily telling Voiceover to ignore touches.
The Scrub Gesture (iOS)
When Voiceover is running on iOS, you can perform what is known as a "scrub" gesture to activate a (properly implemented) back button, if one is present. Try it: turn on Voiceover, open up your Settings app, and double tap on one of the categories. In the upper left corner you will see a 'back" button. Instead of double tapping it, though, place two fingers on the screen and scrub them back and forth, or up and down, a couple times. If you do it right, the button is activated no matter where you perform the gesture. This is similar to iOS7's new "back" gesture for non-Voiceover users. Unfortunately, it is easy to break this functionality.
The Magic
If you implement the standard "back" button, you get the scrub gesture support for free. If you change the button at all, though, or decide to use a different type of control in its place, the button will fail. To support the scrub gesture. You must therefore capture it directly:
-(BOOL) accessibilityPerformEscape {
// Dismiss your view
return YES;
}
As with the Magic Tap gesture, returning "no" will cause this to move up your view hierarchy until it finds something that will respond to it. The up shot of this is that you can use it for more than just a standard back button, such as canceling an operation. However, there is a strong user expectation that the scrub gesture is "back" or "cancel", so it is highly discouraged to implement any other behaviors for this gesture.
Comments
Thanks and keep it coming
the magic
Thank you
This has given me some other ideas for additional VoiceOver features that could be implemented on a future update. Much appreciated.
Detecting if VoiceOver is running on OS X
For the Is VoiceOver Running? section:
I ported the <code>UIAccessibilityVoiceOverStatusChanged</code> notification and <code>UIAccessibilityIsVoiceOverRunning()</code> method to OS X as <code>NSAccessibilityVoiceOverStatusChanged</code> and <code>NSAccessibilityIsVoiceOverRunning()</code>.
You can find the soure code on <a href="https://source.ind.ie/project/voiceover-status-detection/tree/master">the VoiceOver Status Detection git repository at Ind.ie</a>.
Implementing magic Tap support
Hi, I am developer of the app Sea Weather Pro. I am right now working on improving my app to support magic taps. I found an issue under iOS 7 and iOS 8, where views which are presented in a modal way do not allow to capture the magic tap event (e.g. in a sheet on iPad oder as a covering view on the iPhone).
What I recognize instead is that the control center seems to capture the magic tap events and starts playing music. Is this a known or even expected behaviour for voiceover users?
Thank you for this posting, I just soak up everything voiceover to make it work well, but several things seem to be not very easy to implement. E.g. I am struggling with the behaviour of voiceover to automatically focus the top left element on each screen, that that is not really what I think the user expects. Are voiceover users used to this behaviour or are they expecting apps to behave differently?
regards,
Helge
Super Useful: Many thanks!
Hi mehgcap:
I found this post really helpful and wanted to say thanks. It helped me to implement accessibilityPerformEscape which I believe should make a nice improvement in an app I'm working on.
Best,
John
Below is the code for Direct Interaction in iOS swift
Thanks so much for this !
I was looking to implement Direct Interaction (DI) and this was very helpful. Below is the swift code for the DI
self.view.isAccessibilityElement = true
self.view.accessibilityTraits = UIAccessibilityTraitAllowsDirectInteraction
Best
Tarun
Thanks, this has been
Thanks, this has been extremely helpful and I'm now implementing these features into Weather Gods.
Don't forget switch control and state change
appreciate your effort
thanks for this article, but would you please, update it to be usable for latest ios now?
or it's already working?
Web Views and the Scrub Gesture
I work on an app that is a hybrid of native and web views. when web views are instantiated, there is a Close button instead of Back and the scrub gesture doesn't work. (It doesn't work with the Back button either, but that's beside the point.)
Is there a way to have the scrub gesture work when a web view is on screen?
Thanks for the guide.