macOS Beta Testing for the Beginner

Intro

Each year in June, Apple previews the next major version of macOS, along with updates to its other software platforms, at its World Wide Developers Conference (WWDC.) At this event, Apple announces and demonstrates a select number of headlining new features, and makes a prerelease version available to developers so they can test the new software and provide feedback before the public release in October.

This is important because when substantial feature additions and changes are made to an operating system as intricate as macOS, many bugs will also be introduced. This can include issues with how third-party apps and accessories interact with the operating system, making thorough testing by developers crucial.

Even if you’re not a developer, however, you may still be interested in testing the new software to check out the new features or see for yourself what’s improved and what needs work. In this guide, I will give an overview of how best to do this with as few headaches as possible, as well as how to give Apple high quality feedback. Note that as this guide is intended to give a broad overview of macOS beta testing, I won’t be going into detail about specific features or bugs in the operating system.

Terminology

A “Beta” is a piece of software whose features have been conceived and coded to a basic level of operation, but may have issues that can only be identified by a wider pool of testers from outside of the company who bring diverse experiences and use cases to the table. Thus, the more feedback Apple, or any software developer, receives during a beta testing period, the more stable and refined the public release will likely be.

A “Bug” is something in a piece of software that does not work as the user or developer expects, whereas a “feature” refers to a change that’s been intentionally implemented by the developer and is working as they expect it to. For you, as a user and tester, you should report anything you notice that doesn’t work as you expect, as even if a change is intentional, not all changes Apple makes are good, and they may need to hear from users to get a better idea of a given change’s real world ramifications.

Apple’s software development schedule

While Apple’s precise schedule and behaviors are always fluid, they generally stick to a basic pattern for continually refining macOS.

On the day of the WWDC keynote in June, the first developer betas of Apple’s software platforms are released or “Seeded.” As these betas are primarily intended for developers, they are largely intended for testing of third-party apps and accessories for compatibility, and thus many user-facing features may be unstable or may not work as expected.

As development progresses, Apple will seed new iterations or “Builds” of the software that resolve known issues and gradually improve its overall quality. Once they are confident that the software is stable enough for non-developers to test, they make it available to members of its public beta program. As public betas are intended for a wider pool of testers, software released through this channel is generally more mature and stable than that released exclusively to developers, however serious bugs may still be present. Once public testing has begun, builds that have been seeded to developers will typically become available to public testers either the same day or a day or two later, depending on the extent of known and discovered issues in the software, as well as its overall maturity.

Once the beta cycle is complete, usually in October, Apple will seed a release candidate (RC) to both developers and public testers. Barring any critical bugs, this will be the build that is released to the public. Once a version has completed beta testing and has been released to the public, it is referred to as the shipping version.

After a major version has been released to the public, subsequent updates will be released throughout the year to fix bugs and add minor features and enhancements. These updates will also go through a beta testing period for both developers and public testers, although it is usually shorter than the period for major versions. While not set in stone from year to year, planned minor updates are typically released in December, January, March, May, July, and September, rapping up toward the end of the beta cycle for the next major version.

Is macOS beta testing right for me?

As mentioned earlier, beta versions of macOS, or any piece of software for that matter, are often unstable with many bugs and unpredictable behaviors, and are thus not recommended to be used in production environments. Therefore, especially if you rely on your Mac for school, work, personal care, or other essential functions, you may want to install the beta on a separate Mac or volume specifically designated for beta testing. This is especially important for macOS beta testing, as downgrading to the shipping version from a beta can be complicated, and the process to do so will result in the loss of all data on the disk. More detailed instructions on how to do this are given later in this guide.

If you decide to install a macOS beta, you should back up your data beforehand using Time Machine or another backup solution. For more information, check out this guide to backing up your Mac with Time Machine using VoiceOver.

Creating and using a separate volume

If you want to see what the latest version of macOS has to offer but can’t risk the potential complications, you can create a second volume on your Mac’s internal disk to install and run the beta on. This way, you can run the beta when you want, but continue to use your Mac with the current shipping version of macOS. To create a volume to install macOS betas on:

1. Open Disk Utility, located in the Utilities folder (or find it with Spotlight or Siri.)
2. Select your startup disk, usually called Macintosh HD, in the table, and click the “Add APFS volume” button in the toolbar.
3. Name the volume (it can be any name) and if you want to specify a minimum and maximum amount of storage space for it, click size options, define your parameters, and click OK. Otherwise, click Add to create the volume, which will share space with your startup disk.

Once the volume has been created, download the current shipping version of macOS from Apple’s macOS download site, and follow the onscreen instructions to install it. At the point where the installer confirms that macOS will be installed on your current startup disk, click show all disks, and select your newly created volume. Once the installation completes, your new copy of macOS will start as if the Mac was new from the factory; follow the onscreen instructions in the Setup Assistant, then install the beta using the steps described in the next section of this guide.

Note: If you use Time Machine and have your backup disk connected when setting up the new copy of macOS, you may be asked if you want to use it with Time Machine. As your existing volume is already being backed up, I’d recommend opting not to, so as to prevent a beta version of macOS from interacting directly with, and potentially modifying, a backup of your data. Also, you may want to exclude your secondary volume from your backup. To do this with Time Machine, when started up from your primary volume, go to System Settings > General > Time Machine, click Options, and add the secondary volume to the list of excluded items.

To switch between the two volumes, you can either go to System Settings > General > Startup disk to select a default volume that the Mac will start from each time it is restarted or turned on, or select a different volume as your Mac starts. To do this on a Mac with Apple Silicon, shut it down, press and hold the Power button until the startup options dialog appears, interact with the volume you want to start up from, and click Continue. On an Intel-based Mac, restart it or turn it on while holding down the Option key; a list of available volumes will appear on the screen, however there is no audible feedback in this environment. When I had an Intel-based Mac, I found I could hold down the Option key for about five seconds, press the Right-Arrow key once to select my secondary volume, and press Return to continue the boot sequence, however your individual results may vary.

If you no longer want to run the beta on your Mac, you can remove the volume by starting up from your primary volume, opening Disk Utility, selecting your secondary volume in the table, and clicking the “Delete APFS volume” button in the toolbar.

Installing the beta

Obtaining a macOS beta involves signing up for either the developer or public beta program, after which an option will appear in System Settings > General > Software update to install updates from that channel.

Before installing the beta, verify that your Mac is supported by the new version of macOS. Apple’s preview page for new versions, accessible after the WWDC keynote has ended, lists which Mac models are supported by the new version. Once you’ve verified that your Mac is supported, make sure it’s running the latest shipping version of macOS by going to System Settings > General > Software update, and installing any updates if necessary.

Then, go to either developer.apple.com, to obtain the developer beta, or beta.apple.com, to obtain the public beta. You’ll then be prompted to sign in with your Apple ID and agree to a license agreement. Once you’ve signed up, go to System Settings > General > Software update, and click the “Show detail” button to the right of “beta updates.” Select the beta you signed up for in the popup menu and click Done; the beta should then appear like any other update that you can download and install.

Once beta updates have been turned on, Feedback Assistant will become visible on the Dock and in the Utilities folder. In addition to being the portal for testers to report feedback to Apple, Feedback Assistant also contains release notes that list new features, changes, and new and resolved issues in the current beta build. Therefore, you may wish to review these notes before installing the beta, however not all possible issues, changes, and fixes will be documented.

Using the beta and reporting feedback

Now that you’ve installed the beta, it’s time to put it through its paces. Generally, when testing a macOS beta, you can get the best results by attempting to mimic your workflow (minus any critical work) to determine what’s improved, what could be better, and what is broken.

Especially in the early days of a beta cycle, you will likely find many things that don’t work as expected, or changes that simply baffle you, all things you should report to Apple using Feedback Assistant. If you have questions about something you discover in the beta, or are unsure how to clearly describe a bug or suggestion, you can post to the AppleVis forum for help. Ultimately, however, while this may help you understand a feature or describe a bug, reporting your findings directly to Apple provides the best chance that they will be seen and acted upon. Also, because Apple is more likely to prioritize and address bugs and suggestions if they receive a large number of similar reports, you may want to mention on the forum that you reported something, how you did so, and encourage others who think something isn’t right or could be better to do the same.

When submitting a report via Feedback Assistant, you’re asked a series of questions that, along with relevant diagnostic information that is collected and sent automatically as part of the report, should hopefully explain the bug or suggestion to Apple. In addition, including a screen recording can be helpful when reporting a bug, as Apple can see the bug occurring and how it can be reproduced. To do this, press Command-Shift-5, click “Record entire screen,” and in the Options menu, select your audio source (like your Mac’s internal microphone) as well as where you want recordings to be saved. When you’re ready, click Record, reproduce the bug, and click “Stop recording” in the status menu.

Note: For best results, you should make sure Screen Curtain is off before starting your recording, and keep VoiceOver at a comfortable volume and speech rate. Also, to protect your privacy, screen recordings should not contain sensitive information, as they are viewable by a large number of people within Apple.

Feedback Assistant

As mentioned previously, Feedback Assistant, also known as Apple Feedback, is the app used for reporting bugs and suggestions, and exchanging information about such reports with Apple. It is visible on any Mac with Beta updates turned on.

Similar to the layout of the Mail app, the main interface of Feedback Assistant consists of a table of folders, a second table of reports in the selected folder, and the feedback content area, which displays the contents of the currently selected item. Navigate the tables using the up and down arrow keys or first letter navigation, and quickly cycle between tables and panes with VO-J. Alternatively, you can open a report by focusing on it in the table and pressing Return, which will open a new window which only shows the feedback content area for that report.

To create a report, choose File > New feedback, (or press Command-N) select “macOS” in the table, and press Return. Below I describe some of the things you may then be asked, however the specific prompts you see will depend on the nature of your report.

• Problem area: This is where in the operating system the problem occurs. Generally, if you’re reporting a bug with VoiceOver itself, like speech failing or a feature not working as expected, you should select “Accessibility” as the problem area. However, if you’re reporting an accessibility issue within a specific app, like an unlabeled button, you should select that app as the problem area. If you’re reporting an issue in a third-party app, you should also contact that app’s developer to make them aware of the issue.
• Title: This is a short description, usually one sentence in length, that summarizes the report. An example of a title could be something like “VoiceOver crashes when attempting to [do something],” “[something] is sometimes [what it is],” or “the button to [do something] lacks an accessibility label.” If you’re reporting an unlabeled element whose purpose you don’t know, you could title your report something like “Unlabeled button in [app or context within app].”
• Type: This is a categorization of the report, such as incorrect/unexpected behavior, application crash, application slow/unresponsive, or suggestion. Note that a “Crash” is when an app quits unexpectedly and a message is displayed reporting as such. When VoiceOver crashes, it should immediately restart itself and announce “VoiceOver on” however if it remains silent, this should definitely be reported.
• Description: This is where you describe the bug or suggestion in more detail. Below I list some tips to help create a good description:
  • If reporting a bug, describe what you expect to happen in that situation versus what actually happens, as well as steps to reproduce the behavior. If the bug occurs intermittently or inconsistently with no clear cause, mention this in the description and if not asked in other prompts, include the date and time the bug last occurred. Knowing the time an intermittent bug last occurred can help engineers narrow their search of your Mac’s diagnostic information, increasing the chances that the problem will be effectively identified and addressed.
  • If reporting an accessibility bug and it hasn’t been asked in other prompts, information such as the voice or synthesizer in use, presence of voice assets for more than one language, and information about third-party devices in use, like braille displays, may be helpful to include in the description.
  • If reporting a suggestion, describe the current behavior, where it falls short in your use case, and how your suggestion would make it easier or more intuitive.

To attach a screen recording or other file to your report, click “Add attachment,” select “choose file” and locate the file using the open dialog. When you’re ready to submit the report, stop interacting with the scroll area and click Submit.

After your report has been submitted, Apple may request additional information or respond to you if they think a bug has been fixed or suggestion has been taken into account, however as they receive a large number of reports, they can’t respond to every one individually. Generally if Apple responds to a report you submitted, you’ll receive an email notifying you of the message, which you can then view in the “Requests” folder in Feedback Assistant. Once a bug you reported has been fixed, you should let Apple know by selecting the report in the table and choosing File > Close feedback (or pressing Command-K).

Downgrading to the shipping version

Once you’ve installed a macOS beta, reverting to the shipping version on a volume that previously contained a beta can be complicated.

On an Intel-based Mac, the process involves starting up from either Internet Recovery or a bootable macOS installer, completely erasing the volume group in Disk Utility, and reinstalling macOS. On a Mac with Apple Silicon, you must connect it to another Mac and use Apple Configurator to restore the firmware, which will erase all data and reinstall the shipping version of macOS. As I have no direct experience with Apple Configurator, I can’t comment on the practical ease or accessibility of this process, however you can find general instructions in the Apple Support article “Revive or restore a Mac with Apple silicon with Apple Configurator 2.”

Conclusion

As I’ve hopefully articulated in this guide, and as you’ll likely see in your own experience, beta testing can be both an incredibly frustrating and rewarding activity. When testing macOS betas, you witness firsthand Apple’s most questionable design decisions and spectacular coding errors. Conversely, as the beta cycle progresses, you may be able to see how bugs you’ve reported have been fixed or things you’ve suggested have been implemented. More information can be found on Apple’s Developer Program website, Apple’s Beta Program website, and the AppleVis forum, and if you have any questions or believe any of the information in this guide is inaccurate, sound off in the comments.