Apple Vision Pro will propel an AR future
Apple Vision Pro will propel an AR future
Breaking down Apple's new $3,499 spatial computer as efficiently as I can.
I won’t lie: there were a couple of times during Apple’s WWDC23 keynote that my jaw dropped. It wasn’t during the new contact “posters” section of the iOS 17 announcement, nor was it when the M2 Ultra processor was unveiled. It happened toward the end of the show, when CEO Tim Cook announced they had “One More Thing” to discuss.
I’m, of course, talking about the Apple Vision Pro, Apple’s first foray into the world of augmented and virtual reality, spearheaded by a device that lasts up to two hours on a charge and costs $3,500. It’s a very technically impressive headset, one that’s been years in the making and has taken on different shapes and purposes in the process. The final product? A spatial computing device that Apple thinks normal people should own, but no one will likely buy.
It’s $3,499. That’s a lot of money for what Apple seems to believe is a consumer-grade device, when in reality it’s far better suited for enthusiasts and developers. It’s also a first-generation product, and if the Apple Watch is any indication, that might be reason enough not to buy it.
But it’s an exciting product. It’s hard for anyone remotely interested in technology to not be impressed with this thing, what with its powerful computing prowess and futuristic design.
It’s also already being heavily criticized, with many taking it as evidence that a dystopian future where we all wear headsets 24/7 is on the horizon. I’m hesitant to agree with those folks because Apple doesn’t seem too interested in a completely virtual reality. Instead, they want to blend the physical world with the digital, and the Vision Pro is the first step toward that future.
The Apple Vision Pro lives in the real world—most of the time
Look at any demo of the Vision Pro and you’ll see how it blends augmented objects with your surroundings. Take a gander at the front of the headset itself and you may see a pair of eyes looking back.
That’s Apple’s current strategy to keep full immersion away from the Vision Pro as often as possible. Tim Cook said in the past that virtual reality can be good at times, but people probably don’t want to be in it all the time. That’s why Apple has been so focused on making AR work through experiences on iPhone and iPad. And now that they have a headset to toy with, they can create even richer experiences.
That being said, Apple is still willing to give you the choice of being fully engulfed in a digital environment. All of the objects you interact with will blend with your natural surroundings, complete with three-dimensional design and shadow effects. But when you want to shut the world off for a while, there’s a dial that you can crank up and replace your real-world backdrop with a digital one, known as an Immersive Environment.
When you do this, your eyes won’t be visible on the other side, and folks will know you’re busy. But if someone wants to talk to you, they can still appear in the frame as—for lack of better words—a glitch in the matrix; they’ll interrupt whatever you’re doing through a faded window in the headset.
You’re never really shut off from the world with the Vision Pro, which makes it a more interesting concept than something like the Meta Quest 3 which forces you inside a VR experience at all times. And that, to me, is a fascinating approach, one that could ease more people into the world of mixed reality than any device has before.
No shortage of experiences and functionality
Although it seems like it at times with all the company’s marketing jargon, Apple isn’t reinventing the wheel here. The Vision Pro has a strong focus on interacting with normal, everyday apps and giving you a more immersive viewing experience for content, much like other VR headsets. There’s also plenty of Apple flair sprinkled in to give it an edge.
Powered by the new visionOS, the Vision Pro will let you open familiar apps like Safari, Music, Apple TV, and Notes side-by-side on a canvas that’s only limited by the size of your room. There’s a home screen for launching applications, a full-fledged App Store for downloading new ones, and support for Apple Arcade games (over 100 titles will be available at launch). You can also run iPhone and iPad apps.
Apple will be taking a swing at productivity by including support for the Magic Keyboard and Magic Mouse, and you’ll be able to expand your MacBook’s display by simply opening it and looking at it through the headset (it’ll blow your display up to a humongous size with a crisp 4K resolution). Plus, Apple said Microsoft will be offering its suite of Office apps for Vision Pro, which will be a big deal for many who rely on them each day.
FaceTime will also be supported, although it’s a little bit weird. Conversing with others will feel normal; they’ll follow you around wherever you go in floating bubbles, and each bubble will offer a live preview of their real faces. For those on the other end, however, they’ll see your “Persona” which is created using the 3D cameras on the front of the Vision Pro. After registering your face, the device will create a digital version of your head and use it as an avatar while you speak with others. It’ll replicate face and hand movements as you speak to make it seem like you’re using your iPhone or Mac to converse. Apple’s video demo of the feature made it seem pretty easy to set up, and the movements were pretty natural.
Viewing photos will feel more immersive with the Vision Pro (especially panoramas), and you’ll be able to take new 3D photos using the headset’s camera system. Unfortunately, it doesn’t look like you’ll be able to take 3D video, but what are you gonna do.
Entertainment will also be a huge deal on the Vision Pro. Emulating screens that feel 100 feet in width while Spatial Audio music and sound effects surround your skull, there’s no doubt that this will be quite the device to watch movies and TV shows on. (Disney even came on stage during the event to announce they’re working on a few experiences for Vision Pro, and that Disney Plus would be available for the headset on day one.)
There were also a bunch of other demos of what you could do with this headset, including a lot of use cases for education and learning. Beyond pushing the limits of current technologies, Apple doesn’t have a clear goal with the Vision Pro, and that’s okay; it’s throwing everything at the wall to see what sticks, and we’ll no doubt watch a few things fall while others cling on.
An incredibly impressive design and spec sheet
The Vision Pro isn’t $3,500 because you can scan your face and animate yourself over FaceTime calls. There’s a lot of bleeding-edge technology that powers the experience, all crammed into an elegant design that I can only sum up into two words: very Apple.
Design
The headset itself is made of aluminum and a singular piece of “three-dimensionally formed and laminated” glass that melts into the sides and acts as a lens for the cameras and sensors. The Light Seal (a.k.a. the fabric portion that rests on your face) comes in a variety of sizes and shapes to contour to your face, while the Head Band also ships in various sizes and is modular, allowing you to swap it out for a different size or type.
Apple touted that it studied thousands of different heads to make it as versatile and comfortable as possible, and I don’t believe it’ll have any issues fitting snuggly to your face. My main concern will be weight, which is something we don’t know yet.
We also don’t know how hot this thing could get. Apple has a thermal system on the bottom of the headset to increase airflow, but given the amount of power packed in, there’s a good chance it’ll heat up sooner than later.
Power, performance, and controls
Inside the slim headset are two micro-OLED displays (each the size of a postage stamp) with 23 million pixels between both of them, which is wildly impressive to say the least. Apple says they come with “wide color and high dynamic range,” although exact specifications are unclear. In addition, you can snap in Zeiss corrective lenses if you wear glasses using a magnetic connection system.
Powering those screens is an M2 chip coupled with Apple’s new R1 processor. The R1 works to limit lag and increase responsiveness so that you don’t feel motion sickness while using the Vision Pro. The R1 also powers the 12 cameras, five sensors, and six microphones “to ensure that content feels like it is appearing right in front of the user’s eyes, in real time.” In addition, the R1 chip can stream new images to the displays within 12 milliseconds, which is eight times faster than the blink of an eye.
Speakers can be found on either side of the headset (enclosed in the band) with support for Spatial Audio. Your ears will remain completely free if you want to use AirPods instead, although it doesn’t look like you’ll have room for AirPods Max.
Battery life is one of the biggest points of contention for the Vision Pro given how short it is. Apple says it’ll last up to two hours using a bundled battery pack that connects to the headset with a magnetic charging cable. Given the price, you’d expect much more in the endurance department, but I guess this is all Apple was able to squeeze out.
You might be wondering where the controllers are for the Vision Pro. To find them, just look at your hands. Apple’s doing something smart in that it’s not requiring any clunky controllers to be paired with the headset to work; instead, you use your hands to scroll through menus, tap/click buttons and icons, and more. You use your eyes to point at things, then pinch your fingers together to select it. In Apple’s demos, it seemed incredibly intuitive and responsive, but that’s to be expected from a pre-recorded video.
EyeSight, Optic ID, and security
Apple’s EyeSight is the feature that lets your eyes peak through the other side of the display. When you aren’t fully immersed in VR, the Vision Pro will show a live feed of your eyes and make interacting with others feel a bit more natural. If you’re immersed in something, a graphic will appear on the screen and block your eyes.
There’s also Optic ID, Apple’s new security system for scanning your eyes to unlock your Vision Pro. I assume it’s as secure as Face ID or Touch ID, but the jury’s still out on it.
Here’s what Apple had to say about security on the headset.
Apple Vision Pro is built on a strong foundation of privacy and security, and keeps users in control of their data.
Optic ID is a new secure authentication system that analyzes a user’s iris under various invisible LED light exposures, and then compares it to the enrolled Optic ID data that is protected by the Secure Enclave to instantly unlock Apple Vision Pro. A user’s Optic ID data is fully encrypted, is not accessible to apps, and never leaves their device, meaning it is not stored on Apple servers.
Where a user looks stays private while navigating Apple Vision Pro, and eye tracking information is not shared with Apple, third-party apps, or websites. Additionally, data from the camera and other sensors is processed at the system level, so individual apps do not need to see a user’s surroundings to enable spatial experiences. EyeSight also includes a visual indicator that makes it clear to others when a user is capturing a spatial photo or video.
The big picture
A first-generation Apple product that costs way too much money.
We’ve heard this pitch before. The iPhone was considered a luxurious item only for the rich and famous when it launched in 2007, the Mac took years to creep into average consumer’s homes due to its price, and the Apple Watch was unsure of itself until three or four years after Apple launched gold-plated versions of it that cost $17,000.
This is a new category for Apple, and they’ll try their hand at a variety of approaches to figure out what to do with the Vision Pro. Right now, it seems that they want to focus on the idea of “spatial computing,” which is just a fancy way of saying mixed reality. By blending both virtual worlds and augmented objects with the physical world, Apple could get a solid head start on the ultimate goal of a lot of VR companies: a pair of ordinary glasses that can interlope AR objects with the real world.
If it can’t hit a mainstream stride, the headset will at least lay the groundwork for future mixed reality headsets and glasses from the Cupertino company. Apple likely won’t sell a lot of these, and we won’t even know if it’s worth buying until early next year when it launches. But it’s an exciting new area for the iPhone maker to be in, one that could dictate how we communicate with each other after the age of the smartphone.
So, who has $3,500 I can borrow?