GarageBand on iOS: Accessibility case study and call to action

Motivation

There are literally thousands of great things you can do with an iPhone, including making great music. Of course, with Voiceover being the amazing screen reader it is, there is no valid reason why blind musicians shouldn't be able to enjoy these possibilities on par with their sighted peers. After all, Stevie Wonder was featured in an Apple ad at least once, and that in itself sets expectations. So one could reasonably expect, if blind musicians are on Apple's radar, that GarageBand and Voiceover should turn out to be a dream team. The technology exists.

And indeed there are some GarageBand features, like Smart Drums for instance, which feel like they were designed explicitly with accessibility in mind. The overall experience to me, however, is that of a sturdy-looking bridge which begins to crack and strain under your weight just when you have reached the halfway point. It seems as if accessibility was dropped somewhere along the way, with new features getting added without consideration for it, resulting in an inconsistent experience with some good, some bad, and some ugly parts.

Now one might argue that blind musicians using GarageBand are a subset within a subset within a subset, in other words, a very small minority. Yet, isn't that the bread and butter of accessibility? In the final analysis, isn't accessibility about empowering people no matter which subset of a subset they belong to, simply because everybody is worth it?

I strongly urge Apple to put some love and care into the accessibility of GarageBand on iOS again, so that the user experience really lives up to the expectations that have been so carefully set. And if you, the reader of this post, are as passionate about music and accessibility as I am, please do contribute to the process by going to apple.com/feedback and making your voice also be heard. Currently we are missing out on something extremely beautiful with a lot of potential, and it is our own fault if we do not raise awareness of this. I haven't given up on it, and if you have read this far, it appears, neither have you.

With that being said, here, for everyone's reference, is the detailed account of my GarageBand adventures. Let the games begin, and accessibility prevail!

What was not tested

Live Loops and remixing have been excluded from this test as those rely heavily on hand-eye coordination. They should be addressed only after more basic issues listed below have been corrected.

I have also not tested amps as I am currently lacking the necessary hardware, an unfortunate omission.

Adding a track

In the interface for adding a new track, there is a "More Sounds" button on each page. This is showing the following behaviors:

• Critical: On the drums page, it switches to a part of the interface which provides no Voiceover feedback at all. Only a reset of GarageBand in Settings will once again render the app usable.
• On all other pages, nothing happens.

The sound library page contains only one element which Voiceover records as the number 9. When double-tapped, this does nothing. Double-tapping and holding also does nothing.

Documentation bug: After a track has been added, there is a button on the left side of the control bar for adding more tracks. The user guide refers to it as the Browser button, whereas Voiceover refers to it as instruments and live loops. I strongly feel that the nomenclature in the user guide should match the one exposed to Voiceover.

The menu button

Just a minor documentation bug: The button that in most scenarios occupies the top left corner is referred to as the Navigation button, whereas Voiceover calls it menu.

Playing a touch instrument

Critical: When chord strips are shown instead of notes, the keyboard is unreliable with Voiceover. When turning Voiceover off, the chords play as expected when touched.

Documentation bug: There is a button for showing and hiding the controls area. The user guide refers to it as the Controls button, whereas Voiceover calls it instrument settings.

Very minor inconsistency: The user guide refers to Octave Up and Octave Down buttons. Voiceover sees this as a single adjustable. This is accessible as-is and should simply be corrected in the user guide.

On a similar note, the keyboard mode button is described in the user guide as having the settings glissando, scroll, and pitch, whereas Voiceover sees those as normal, scrolling, and pitch bend, respectively. Also, the effect described by the user guide as glissando would be better described by the word legato.

Instrument controls

This section describes accessibility challenges encountered in the control areas of different instruments. By control area I am referring to the assortment of knobs and similar elements used for adjusting instrument parameters. Please note that this list is incomplete: I am not going through all of them, just taking samples.

Classic rock organ and heavy metal organ

The control area of this beautiful Hammond emulation is, alas, completely inaccessible. To a Voiceover user, its constituent controls are rendered as:

• An unnamed adjustable
• An element of unknown type named "3RD"
• An element of unknown type named "on"
• An element of unknown type named "off"
• A button named "hammond viewswitcher drawbars"
• An element of unknown type named "2ND"
• An element of unknown type named "percussion"
• An element of unknown type named "chorus"
• An element of unknown type named "distortion"

Note that I am observing the above elements switching their swiping order for no apparent reason, but I believe I have listed them all.

Electric pianos and most synths

This is an example of accessibility done right. All sliders seem to be clearly named and react to gestures as expected. I am including this positive example here to stress the fact that those control areas can, in theory, be accessible, and should be in practice.

Alchemy synths

The snapshots within the transform pad activate when double-tapped, but Voiceover does not recognize them as buttons. The framing box is not announced at all, so dragging it is impossible with Voiceover. This means that the versatility of the Alchemy Synth is mostly lost in translation. Also, it isn't clear to me how to identify the two pucks within the XY pads.

Sampler

Editing the shape of a sample is tricky as the control points will not announce their current value. My suggestion: Have the control points utter their value as percentage when located, and also when dragged.

Trimming is tricky for the same reason: The resize handles do not announce their positions. Suggestion: Have the handles announce their positions in some unit of time when located and as dragged.

Playing the drums

The drums have a tendency to cover the screen in such a way as to make it hard for a Voiceover user to activate other controls, such as the menu button, or to select a category and then a drum kit from the respective lists.

Also, the drums do not behave reliably with Voiceover, sometimes announcing their names, then at other times playing their respective sounds. Direct Touch seems not to know exactly when to trigger.

Smart drums

Congratulations, those are amazingly accessible. They suffer ever so slightly from the nomenclature problem, with buttons not matching their descriptions in the user guide, but from the respective context a correct deduction can always be made. I especially love how the parameters (loudness and complexity) are announced when dragging a drum (somewhat loud, complex, somewhat soft, etc.). Let this serve as an example illustrating that even visually complex scenarios can, and should, be rendered accessible.

Beat sequencer

This seems completely inaccessible. There are controls recognized by Voiceover as follows:

• Button SS Header Mute
• Button SS Header Solo
• These two alternate repeating 6 times
• Button SS Power Button Off
• Button SS Pattern Library Off
• Button SS Dice 5
• Button SS Settings Off
• Untyped Step On/Off
• Untyped Velocity
• Button SS Triangle

The library button opens a popup of available patterns but they are all announced as dimmed and will therefore not react to double-tapping.

Clearly whoever did the smart drums ui should have a look at the beat sequencer. :)

Audio recorder (SEVERE)

The preset dial in Fun View exposes no way for Voiceover to interact with it. It announces the name of the currently selected preset, but it does not react meaningfully to any gestures. By double-tapping and holding it can be modified but it provides no corresponding feedback as it turns, so the number of presets skipped cannot be determined except with a lot of trial and error.

The additional knobs associated with each of the presets just announce their names, such as "size" and "space" for the monster preset, but there is no way to modify them or even to read their current values with Voiceover. The same goes for all the knobs in Studio View.

Note: Despite the brevity of this section, the point it makes cannot be overstated. The present accessibility issues are confining Voiceover users to only the most basic recording options.

Playing strings

When playing individual notes, the articulation button and the instrument selector are not announced and are therefore tricky to locate. Also, it seems to be possible to lock the articulation button, resulting in permanent pizzicato and bowed playing rather than legato. This possibility should be described in the user guide.

Virtual drummer

The button for deleting a drummer region is, confusingly, called "arrange region drummer deleteb."

Playing world instruments

Please note that I have skipped testing world instruments as I am not versed in their theory. My knowledge of music does not extend far beyond western concepts. As I familiarize myself with other musical traditions I shall test the Asian and African instruments here as well.

Track controls

This section refers to the track controls reached by tapping settings then track controls from within an instrument view.

Critical: The track control volume slider does not lend itself to any precise adjustment with Voiceover except all the way down or all the way up, announced as 0 and 1 respectively. The same goes for the compressor, as well as echo and reverb under master effects.

Tracks view

Using track headers

The user guide mentions dragging a track icon to the right in order to show track headers, including a mute button, a solo button, and a volume slider. With Voiceover this action does nothing, so the controls mentioned here are only available by tapping settings then track controls, which may be tedious if multiple tracks are involved in a complex editing operation.

Navigating

• The user guide mentions double-tapping a position on the ruler to quickly go to that position within the song. I cannot deduce a Voiceover equivalent. Please advise.
• The user guide also mentions scrolling through a track by swiping left or right across an empty section of the screen. Please advise as to the Voiceover equivalent. Please also advise as to how scrolling would work if the relevant screen section were completely covered by regions.
• Zooming, according to the user guide, is accomplished by pinching open and closed. Since Voiceover reserves the pinching gestures for its own selection operations, these do not work in the given context. Please advise.

I would be willing to perform more extensive testing of these aspects but to do so I require the missing information listed above.

Selecting regions (SEVERE)

Selecting multiple regions is supposedly accomplished by touching and holding one of them, then, while still holding it, touching additional regions to select them along with the first one. With Voiceover, touching and holding would amount to double-tapping and holding. I can perform this on a region but then Voiceover does not provide feedback for the second finger trying to discover additional regions for selecting. Beyond missing feedback, multi-selection seems not to be happening in the first place. Despite multiple attempts I could not make it happen.

By tapping a track header I am able to select all regions within that track. That would be a good starting point for multi-selection, but then tapping specific regions does not unselect them as described in the user guide.

I have marked this section as severe for the following reason: Editing anything with confidence requires precise control over what is selected, that is, to what exactly the next editing operation would, or would not, apply. Without this precision, editing becomes a trial and error operation in which the user spends time fighting the interface rather than producing music.

Note editor (SEVERE)

The note editor seems completely inaccessible. Nowhere could Voiceover detect anything that would pertain to modifying the characteristics of a note. I have rated this as severe because a skilled musician will always, sooner or later, see the need for making modifications at this level of granularity.

Region settings

The speed of a region is exposed to Voiceover as an adjustable, but this can only be set to two values called 2 and -2. Please note that this does not refer to "tempo" but specifically to "speed" within a region's settings. All other settings seem to be accessible.

Automation

The automation editor seems completely inaccessible. Voiceover does not find anything pertaining to automation points and their parameters.

Song sections

Changing the order of song sections seems impossible with Voiceover, as the dragging pattern is not recognized. I feel this may be critical for some projects but I can't currently back this up with any evidence.

Conclusion

Well this wraps it up as far as my GarageBand experiments are concerned. There may be more issues lurking which I simply didn't find, and indeed some of the issues I describe may just be user mistakes on my part. I have no way to check, as Voiceover-specific documentation for GarageBand is lacking---perhaps an issue in itself. Let me conclude thus: I applaud the GarageBand team for making the effort to implement partial accessibility. Now let's give our feedback so we may see the rest of the show.