Apple brings augmented reality (AR) to your phone with Measure, the first native AR app in iOS 12.
Apple's Measure app is visually compelling for sighted users, but also reasonably accessible for blind users. This blog describes taking measurements as a blind or low vision user, and enumerates improvements for Apple's consideration to make the app easier to use.
Keep in mind the Measure app is not available on all iOS devices. According to Apple Support, you must have an iPhone SE, iPhone 6s or later, iPad (5th Generation), or iPad Pro. The app installs with iOS 12 on any supported device.
What is AR?
While Virtual Reality (VR) displays scenes rendered with 100% computer graphics, AR seamlessly integrates computer generated elements with a real world camera image.
AR has been used in televised sports for many years. Players appear to run across graphic data painted on the playing field, such as first down distance in American football and lead-off distance in baseball. In golf, players and fans walk beneath an AR graphic arc of the last shot or over a numeric distance seemingly painted on the fairway.
It's an eye-catching technique and sexy way to graphically display information. Unfortunately, its visual nature presents problems for the AppleVis community.
Using the Measure App
Before trying to use the app, read through this MacWorld article for an overview of measuring distances and the level feature. After reading it, come back to this blog for a discussion of accessible usage.
Note that even sighted users are advised to use a tape measure for accurate measurements. The MacWorld article puts it like this.
Finally, remember that Measure is meant to give you a sort of ballpark figure. It might suffice for those "will this object fit on this shelf" moments, but if you need real precision, or if it's a close call, you'll want to break out the tape measure to be sure.
The article fails to describe changing units in Settings. On my phone, Measure defaults to Imperial units (feet and inches). Set Metric units by going to Measure in Settings. To find it quickly, enter Measure in the Settings search box. Handy if you live somewhere other than Myanmar, Liberia, or the United States (the three remaining non-metric holdouts),
Linear Distance
Measuring linear distance requires you to place two AR graphic points in the scene, a seemingly impossible task for blind or visually impaired users. If you can feel two endpoints, you can use Measure to obtain a ballpark distance. Here are some example use cases and tips.
- Distance between any two tactile points
- Touch the point with one hand and aim the camera with the other.
- Double tap the Add A Point button when the app announces Add A Point.
- Distance between two corners
- Aim the camera by touching the corner with one hand, but remove your hand so that the app can recognize the corner.
- When VoiceOver announces Snap To Corner, double tap the Add A Point button.
- The snap tolerance is small. This technique requires patience and practice.
Measuring Between Edges
Unfortunately there is no Snap To Edge feature.
Keep in mind the app can't focus on distant surfaces (more than 7 feet or 2 meters). Also, after adding the first point, VoiceOver interactively reads the distance as you move the camera towards the second point. Use this knowledge of the background surface and the interactive distance to accurately find edges of a surface.
There are two cases to consider.
- If the background surface is distant, VoiceOver announces Lost Focus or Find A Nearby Surface To Measure. Slowly aim the camera towards the target surface until you hear VoiceOver announce Add A Point, which tells you the camera has found a nearby surface, then add the first point.
- If the background surface is close enough for the app to focus on, create a discard line to find the first point. Here's how.
- Place a point anywhere on the background surface.
- Slowly aim the camera toward the closer target surface. When the camera finds the target surface, VoiceOver should announce a large jump in distance. Immediately double tap the Add a Point button. The line you just created is a discard line, but the second point is very close to the edge of the target surface.
- With the camera still pointed at the second point, you should hear VoiceOver announce Snap To Point. Double tap the Add A Point button to start a new measurement, with the first point at the same location as the discard line's second point.
Regardless of whether you have a close or a distance background surface, add a second point by listening to the interactive distance as you move the camera. If you move off the edge of the surface, VoiceOver will announce a large change in distance. Aim the camera back towards the edge, then add the second point.
If you know of other tricks for finding surface edges, please post a comment.
Tips for Linear Distance
Using the Snap To Corner method gives you the best precision the app has to offer, as precise as a sighted person could obtain. With this technique, the Measure app measured a kitchen countertop at 50" or 127cm. The actual distance with a measuring tape is 51" or 129cm.
After adding two points, VoiceOver announces the distance between them. Visually, the app displays a line between the points in the camera scene with the distance in a text box near the middle of that line. To review the metric, point the camera towards the points you added, then swipe left from the Add A Point button or drag your finger on the screen.
When discussing the Measure app with a sighted friend, keep in mind that the Add A Point button appears visually as a plus sign.
With VoiceOver disabled, you can adjust point locations by touching the point, holding, and dragging to the new location. If you know how to perform this operation in VoiceOver, please post a comment.
Rectangles
This feature works well for measuring framed artwork on a wall and objects such as iPads, bluetooth keyboards, and books.
Point your camera at any rectangle-shaped object. When the app recognizes the rectangle, VoiceOver announces Rectangle Detected. Double tap the Add A Point button. The app provides width, height, and area metrics.
If you can't detect a rectangle, try placing your object on a different surface to increase contrast. The app doesn't recognize a white sheet of paper on a white tablecloth, for example. Also try changing the camera distance.
The Add A Point button label doesn't change for rectangles. Don't let this confuse you. If VoiceOver announces Rectangle Detected, Add A Point adds a rectangle, regardless of the VoiceOver button label text.
To review the measurements, point at the rectangular object, then swipe left from Add A Point or drag your finger on the screen. VoiceOver should read the length of both sides and the area.
The app sporadically provides only one side length. The steps to reproduce this issue are unclear. If this problem occurs, double tap the Clear button, and add the rectangle a second time.
Saving the Measurements
After measuring a linear distance or detecting a rectangle, the measurements are displayed as text boxes in the scene. As long as you point your camera at the measured object and drag your finger on the screen to find these text boxes, VoiceOver reads the measurement. Double tap and hold to bring up a menu with an option to copy the measurement to the pasteboard.
Sighted or low vision users might want to take a screenshot containing the measurements. Swipe right from the Add A Point button to find the Take A Screenshot button. This is more convenient than the hardware button screenshot trigger.
Level
The app also works as a level. Double tap the level tab in the bottom right corner.
Use the level in two ways.
Face Up—Lay the phone with the screen facing up to test for level.
Vertical—Hold the phone vertically.
The app switches between Face Up and Vertical modes automatically based on phone orientation. The app displays level degrees text in the center of the screen.
You can also hold the phone at any orientation and double tap to calibrate to zero degrees. Use this to test surfaces with arbitrary orientation.
For sighted and low-vision users, the app display indicates the direction of deviance from level or vertical, but this information is not accessible to blind users.
General Tips
For best results, try these tips.
The app doesn't work in the dark. Turn on any available lighting.
When adding points or rectangles, keep VoiceOver focus on the Add A Point button so you can double tap as needed.
When measuring linear distance, try to find opportunities to use the Snap To Corner feature. Finding a corner takes some patience and practice, but produces superior accuracy.
The app is sensitive to camera distance. If VoiceOver announces Find a Nearby Surface to Measure, or Lost Focus, change the camera distance.
Low Vision Users
Low vision VoiceOver users should find the app easy to use.
Non-VoiceOver low vision users will find the app challenging. The app ignores accessibility settings for Bold Text, Increase Contrast, and Invert Colors, and there is no way to increase the AR point size or line width. As a result, the AR graphics seem to disappear against the underlying camera image.
The app color scheme is inconsistent. Message text appears as light on dark, while measurements and the copy menu use dark on light. Even with support for Invert Colors, this inconsistency would impede low vision use.
Fortunately, the app honors the larger text setting, which might be helpful for some low vision users.
Areas for Improvement
Apple has done an incredible job of making the Measure app accessible. Nonetheless, there's always room for improvement.
For blind users:
Add a Snap To Edge feature. Provide controls to increase the snap tolerance for corners and edges.
Change the Add A Point button VoiceOver label to simply read Add. This eliminates confusion when adding a rectangle and facilitates communication between sighted and blind users.
In Level mode, blind users should be able to access the direction of deviance from level or vertical.
For low vision users:
Add support for Bold Text and Invert Colors.
Use color schemes consistently, either dark text on light or light text on dark.
Outline AR graphics in a contrasting color, such as a black border around a white line, to help it stand out from a light-colored background image.
Add controls to increase AR point size and line width.
Conclusion
The app is a challenge for blind users, but not completely inaccessible. Non-VoiceOver low vision users might find the lack of contrast a serious impediment. Apple can make the app more accessible with some simple improvements.
While challenging to use, Apple has set a high bar for AR app accessibility. Hopefully other app developers will follow suit.