Summary of WWDC 2016: Year of the Software

WWDC 2016

Welcome back to Applevis' yearly summary of Apple's yearly developer conference announcements. There's a lot this year, much of it exciting, some of it very much expected, and a few things long overdue. Instead of a long introduction, let's just start moving!

watchOS

First up is the Apple Watch. Love it or hate it, Apple is committed to it and feels it's a great way to get people more active and connected. To that end, watchOS3 will bring a huge array of updates to your wrist, all of them benefiting from the biggest one of all: speed. Apps launch instantly (well, seven times faster than in watchOS2, but who's counting), and we can expect speed improvements across the entire platform. This is thanks to multiple under-the-hood changes, but mostly due to apps updating themselves in the background, always ready for you to open them. Apple is so confident in this new scheme that you can swipe from app to app, without opening them, and view critical data in real time.

Swiping from app to app? Yep, Apple has added a "dock" to the Watch, which comes up when you press what used to be the Friends Button. What happened to your list of friends is unknown at this point. You can choose which apps are available here, letting you customize a subset of apps you use most, ready to be opened at the push of a button.

Other interface changes coming in watchOS3 include a control center, accessed by swiping up from the bottom of the screen, just like iOS. What's in here, or where your glances went, wasn't explained. Apple has also added more watch faces, including a Minnie Mouse face, a face centered on your activity rings, and several numerical ones. It was also suggested that they've allowed for more complications on different faces. To change faces, you need only swipe between them, rather than force-touching, scrolling, and tapping the face you want. Apple's goal seems to be to make changing your Watch face easier, as their data suggests most users like to do this, and to give people quicker access to more complications. After all, if you like the Astronomy Face but want quick access to a lot of complications, you can just put Astronomy and Modular next to each other, then swipe between them. We don't yet know for sure, but I suspect this is as simple as a two-finger swipe left or right for VO users. After all, that's how you scroll vertically already.

The other big interface change is in messages: no longer do you have to tap the Reply button before you can reply to anything. Now, when you see a message, you already have access to quick replies and (presumably, though it wasn't stated) the Dictate button. I'll be eagerly waiting to see if this means VO users can do a two-finger double tap to dictate a reply on the same screen as the incoming message--ow great would that be? Sighted users can also use handwriting, by writing with a finger, and each letter they draw is recognized and added to their reply. No word yet on whether this indicates a handwriting mode for VoiceOver on the Apple Watch.

Apple has added something rarely used, but hugely important, to the Watch: SOS. Hold in the side button on your Watch and you get a countdown. At the end of that countdown, if you don't cancel, you're suddenly calling your area's emergency number. Once the call is complete, any contacts you previously designated will be notified that you're in trouble, then your medical information (another new addition to WatchOS) will appear on your watch screen for emergency responders to see. All this happens with an awareness of where you are; if you don't know that the UK uses 999 as its emergency number, it doesn't matter--your Watch knows.

In the fitness and health space, Apple continues to improve. The Activity App now offers competition, if you want it. Once you and a friend share information, you can see each other's activity rings, heart rate, step count, and more. Check in and see if you've beaten your buddy's exercise amount for the day, or look at where you stand (pun intended) among the step counts of your friends and family. You can even message the people you're competing with right from the Activity app, and the canned phrases here will be specific to workouts and motivation.

Speaking of workouts and motivation, Apple has set its universal access sights on wheelchair users. With a single setting change, you can set your activity to wheelchair mode. The reminders that it's "time to stand" turn into "time to roll", and the algorithms that determine activity are changed to ones specific to wheelchair users. Apple teamed up with two organizations dedicated to fitness for wheelchair users, plus conducted extensive studies on manual wheelchair movement techniques, to get this right. It's a great thing to see such dedication to another group of disabled people.

Finally, Apple has focused on helping everyone with something we all do: breathing. A new "Breathe" app comes standard with watchOS3, meant to guide the user through deep-breathing exercises. You set the duration (one to five minutes), and the Watch does the rest. You can even use this app with haptic feedback instead of visual aids, as it's meant to be used by people with their eyes closed. If you want to, you can also set up reminders to do deep-breathing exercises, similar t other activity reminders you already use.

All these improvements, plus the addition of tons of new options for developers, make watchOS3 a major release any Watch owner should be eagerly anticipating. It's faster, offers even more activity options, and will let apps really grow and transform. It will be a free update, as usual, available this fall.

Apple TV

The latest Apple TV, the fourth generation, was released last year. Don't worry, no new hardware is coming to replace it (yet). New software, though, will make it an even better platform, though it didn't get as much attention as the rest of Apple's products.

The Remote app for iOS is getting a major overhaul. Primarily, it will let you use your iOS device as a complete Siri remote: voice input for Siri, the touch surface, buttons, and motion tracking for game play are all duplicated in the app.

Siri is getting smarter about finding content, with the ability to search by topic. The example used during the keynote was "find high school comedies from the eighties". The presenter then said "just the family friendly ones" to filter the results. That, plus Siri's ability to search Youtube, will make an already great TV experience that much better when tvOS10 lands this fall. You'll even get single sign-on for cable network apps, letting you sign in just once to one app and instantly authorize others, and a dark mode for your TV.

The Mac Operating System

OS X has been around for over fifteen years, according to the presentation. Yet, when put alongside watchOS, iOS, and tvOS, it stands out. Apple has therefore decided to rename it: macOS. Interestingly, along with the new name came a change I never expected: no number. Not "new number", but "no number". The next version of macOS will be dubbed Sierra, but that's it. We have iOS10, watchOS3 and macOS Sierra.

No matter what you call it, Sierra has a lot to offer Mac users. Let's get the minor changes out of the way first: picture-in-picture for videos, and automatic tabs for all multi-window apps, are both included. What this will mean for VoiceOver users isn't known, but given my own experience with tabs in Finder, I suspect it won't have much of an impact.

Now, onto the fun stuff. First, the great MacID app has been Sherlocked; Apple is now letting you unlock your Mac with your Apple Watch. There was mention of careful planning to make sure it unlocks at the right time, rather than unlocking when you happen to be nearby but still want to keep it locked to the people around you. Presumably, then, you won't even need to confirm anything on your Watch, just be near enough to the Mac while wearing the Watch.

Next up is Apple Pay, which is being brought to the internet. Sites that choose to can add a "pay with Apple Pay" button which will, when you click it, ask for confirmation from you. This can be done either with your Apple Watch (which, unlike unlocking a Mac, you will have to tap to confirm) or Touch ID on an iOS device. As soon as you okay the payment, you've bought your items on the website. No typing card information, or storing your details with a site that could get hacked; you can use the same security of Apple Pay you enjoy at physical locations, but now online. Of course, companies will have to add this to their websites, so let's hope they do. The day I can check out at Amazon with Apple Pay will be a happy day for me!

In keeping with all this connectedness, Apple is making iCloud even more useful. Your clipboard will be shared among your devices, for example. Note that I said 'devices', not 'Macs'. If you copy something on your Mac, then paste on your iPad, what the iPad pastes will be what the mac copied, and vice versa. This not only includes text, but links, images, and other rich content. One iCloud feature that is Mac-specific is desktop syncing. The presentation was a bit hard to follow, but it seems that what's on the desktop of any Mac will mirror itself on any other Mac you sign into. Presumably, this will require you to have the space in iCloud to store everything, and no, new iCloud storage pricing was not among today's announcements.

The last thing to mention in this vein is iCloud's ability to store things you may not need, alongside Sierra's ability to clear out things you can get rid of. For instance, iCloud will hold documents you haven't accessed in a long time, getting them off your hard drive, while Sierra will clear old caches, empty the trash, and perform other tasks. All this adds up, and can remove huge amounts of unused information from a Mac. The example given on stage was of a Mac going from 20GB free to 150GB free. Of course, your iCloud will need to be able to handle the offloaded data.

I've saved (arguably) the best feature for last. At long last, and a year after Microsoft tried it, Siri lands on your Mac! All the commands from iOS you already use seem to be present--messaging, asking for information, playing music, and the like. With access to your files, though, Siri can perform complex searches and list the files you want. When a file/folder list is displayed as a result of a Siri query, you can ask to have it filtered further. Request "documents I worked on yesterday", for instance, and you might then want to say, "only the powerpoint files" or, "the ones about the trip". You can even pin these searches to your Notification Center for quick access at a later point. If you use Siri to pull up a list of files--from all over your hard drive--that you need to work on, pin it, and you can get at that list at any time without having to ask Siri for it again.

iOS10

Finally, we get to the release that will probably impact the most people: iOS10.. There are a lot of features in here (unsurprisingly, Apple detailed ten of them). I'll group some together, but I'll hit all the points we saw today. Speaking of seeing the major points, check out this audio-described iOS10 video from Apple, marking the first time I've ever seen a described product video from any company.

One of the places many of us spend a lot of time is on the lock screen. Knowing this, and knowing that a lot of people find it too easy to accidentally unlock their phones with the faster Touch ID, Apple has copied a feature from their Watch: raise to view. Your phone will detect when you're holding it up to look at it, and will show you the lock screen without you having to press any buttons at all. Once on that lock screen, you'll notice much richer, more interactive notifications. The 3D Touch feature on the iPhone 6s will shine here, letting you look at full calendar views of your day for calendar events, or an entire thread for a message, and so on. Unsurprisingly, developers can customize this however they want to. Depending on how apps are rewritten, lock screen notifications could become far more useful and information-packed than they are right now.

Unlock your phone and you'll find something Android users have had for a long time: widgets. This part of the presentation wasn't entirely clear to me, but it seems that apps can customize what happens when you 3D Touch on them. Instead of a menu of shortcuts actions, you can now get anything the developer wants to offer--recent sports scores, your activity for the day, upcoming calendar events, you get the idea. You can also add an app's widget to your Notification Center, which seems to be a way of getting at the content without having to 3D Touch it. Again, take this with a bit of salt, as the verbal presentation of this feature wasn't straightforward to me.

Next we have the Messages app. It got major upgrades in the form of messaging options: full-screen animations, handwriting and digital drawing, marking up a picture taken (within the app) with the device's camera, and even sending an 'invisible' message that won't appear until the recipient opens it. It seems the basic interface remains the same, there are just a lot more options for how to talk now. The experience was made richer in a lot of ways, such as links now showing webpage previews instead of just URLs or Apple Music links letting you play a song right in the message thread. Emojis were also featured: they're three times bigger, and iOS will offer the option of replacing full words with their emoji counterparts. Plus, Apple has opened the app for extensions written by third-party developers. Reserve a table, order some food, send a Square Cash payment, and so on, all within iMessages. This will happen as apps come onboard, much like sharing extensions did in iOS8.

Similarly, the Maps app has gotten better. The interface is easier to use, with smart suggestions, category filtering, and its own extensions for programmers to use. You could reserve a table and then call a Lyft to get you there, all without leaving Maps. For those who drive, Maps can now try to route you around heavy traffic, and will zoom in and out based on your speed and location. Cruising down a highway, you'll see more of the map than when you're moving through a city, for example.

QuickType, the feature that displays word suggestions above your keyboard as you type, is getting a whole lot smarter. The same context awareness and deep learning that drives Siri is being tied into QuickType, so that it can do a far better job of providing suggestions. As you're asking your friend over iMessage where you should meet for dinner, you might see QuickType suggest a restaurant you were looking at in Maps, for instance. Or, it could tell the difference between your responding to "where did you go yesterday" and "where did you put my bag", offering different responses based on the question. It can even offer word options in two languages, if you frequently switch from one language to another.

The Phone app has gotten some much-needed attention in iOS10. A feature that may be especially exciting to deaf-blind users is your phone's ability to provide voicemail transcriptions. Imagine being able to read your voicemails on a braille display! Apple has also extended the phone handling APIs to third-party apps. What that means for you is some cool stuff: VOIP apps can make themselves act like phone apps. See recent Skype calls in the Recents tab of your Phone app, or call someone using Facebook Messenger's voice call feature right from Phone. Add a VOIP app option to a contact, just like you already can with phone or FaceTime. There's even a chance that Skype or other apps can trigger the same "ringing" we're used to with the Phone app itself, better supporting interrupting other apps or ringing longer until you answer. The details of this integration will be explained to developers as WWDC continues, and I don't think it'll be long after iOS10's release that we start to see apps taking advantage of the new capabilities.

Siri, first released in 2011 and now serving over two billion requests per week, is long overdue for an upgrade. Apple announced just such an upgrade today: a Siri API. Finally, apps can hook into Siri; you could request a Lyft from your current location, order some food, book a table, and more, all from Siri itself. As with other third-party development offerings, this will all depend on apps being updated to take advantage, but I don't think that'll be a problem. Prefer Facebook Messenger? Just tell Siri to "use messenger to tell Bill I'll be late tomorrow." Or say, "have A Lyft get me a ride home", or "send $10 to Jim with Square". We don't know exactly how the phrasing will work, but we do know that any app that wants to can take advantage of Siri's contextual awareness and speech recognition.

iOS will also gain the ability to perform image recognition, without even talking to any remote servers--it happens right on your device. The concept is similar to Facebook's recent auto-descriptions for pictures, but with the ability to better pick out objects. Essentially, iOS will now "look at" your photos (Apple claims iOS performs eleven billion calculations per photo to do this). It can detect faces and match them to contacts, letting you ask Siri for "pictures of Wendy" or tap a contact below a picture to interact with the person. More excitingly, though, it can pick out objects in photos, like houses or trees. Apple has added a feature to the Photos app called Memories, which can intelligently group pictures together based on subject, location, time, and other factors. The image recognition they unveiled today is used to make this happen, as well as to do some cool things like automatically generate a slideshow of a memory. My hope, though, is that they use it to generate descriptions of pictures for VoiceOver users. They didn't say anything about that, but it seems like an obvious application of the technology. Here's hoping!

Apple's Music app has been redesigned. You can easily see which songs you have downloaded; look at song lyrics; play shows from Beats One on demand; and more. It sounds like Apple has made the app easier to use, while still offering the full Apple Music experience.

The News app has, Apple reports, over two thousand publications on offer. In iOS10, this app gets more useful in three ways. First, subscriptions will be supported, meaning that users who pay for content from a paper or journal will get their paid content right in the News app. Second, there are now categories--some set, some automatically generated based on your usage--for stories. Third, News can now give you breaking news notifications if you want it to, which will make it a far more useful app for many people.

There are a bunch of smaller, but no less important, features: 3D Touch to bring up "clear all" in the Notification Center, like on the Apple Watch; an official Home app to control HomeKit devices and activate groups of settings at a touch or based on your location; the ability to collaborate with others on notes; displaying two websites at once in Safari on iPad; and a great deal more.

Finally, Apple was careful to point out how seriously it takes users' privacy. Despite facial and object recognition in photos, extensive app integration with Siri and other core apps, predictive typing with locations and other contexts, and all the other features iOS offers, you aren't a profile. Apple has done a lot of work to be sure that no one can ever be tied to any data they send to the cloud. Apple can get trends and other metrics, while never using your specific data to do it. They also put encryption and security first--Apple Pay, Touch ID, iMessages, FaceTime, and other services are all as secure as Apple can make them.

Bonus: Learn to Code for Free

Some people have suspected that the iPad Air 2, and later the Pro, would bring Xcode (the program developers use to make apps for Apple products) to iOS. That didn't happen, but we got something equally cool: a free app that teaches you how to code. It shows you the basics of controlling what happens when in your program, how to write basic code, and so on. It also offers advanced lessons including physics modeling, access to the iOS SDK, and more. It's a great idea, and will, Apple hopes, teach more students--and even adults--the basics of coding. The app is called Swift Playgrounds, and will be launched this fall, for free, alongside iOS10. It only runs on the iPad, and we have no word on its accessibility at this time.

That's a Wrap

And that's WWDC 2016. macOS Sierra, iOS10, watchOS3, tvOS10, and Swift Playgrounds. There were no hardware announcements, just software, and there sure was a lot of that! Some people are excited, others are grumbling about how long many of these features took, and others are just underwhelmed. Let us know what you're thinking in the comments, and remember: the rules haven't changed this year, so mum's the word about what you find in the betas, even if you're a public tester. Now: Surprised, stoked, sad, or seething... What do you think?