Apple has just announced its brand new product, the Vision Pro mixed reality headset, at WWDC 2023.
It is Apple’s most ambitious product since the iPhone. We are excited to share one of the first three Chinese media/bloggers to try it on today. Since recording was not allowed during the trial, we created this post to convey her firsthand experience and clarify things for everyone.
Introduction to Vision Pro and Initial Setup Process
Vision Pro is an all-in-one AR/VR headset. It presents a world that can be both immersive virtual reality and augmented reality, allowing users to see the external environment. It operates independently without the need for an iPhone or Mac, running on the Vision OS and dedicated apps.
The presentation did not include the initial setup process after receiving the headset, which piqued my curiosity. When I arrived at the trial area, two staff members approached me to gather some information.
Firstly, one of them scanned my face with an iPhone, similar to the Face ID process. They asked me to turn left, right, raise and lower my head to find the right facial shape for the mask attached to the entire headset.
Both the mask and strap of the headset are modular and replaceable.
Next, they continued to scan the contours of my ears with the iPhone to enhance the spatial audio effect. Additionally, an on-site vision test was conducted.
Even I had nearsightedness, they could customize prescription lenses specifically for Vision Pro, which could be easily installed in the glasses. Once they fitted me with the appropriate mask, I was ready to dive into the experience.
Hands-Free, Gesture and Voice-Controlled
The Vision Pro VR device eliminates the need for any handheld controllers and relies entirely on three control methods: eye gaze, gestures, and voice commands.
The staff initially taught the basic gestures, such as lightly pinching to confirm selections, pinching and dragging to move left or right, and wrist flicking for scrolling up and down.
The device incorporates Apple-like details, where flicking the wrist generates an initial velocity, similar to scrolling on an iPhone, enabling smooth and fluid animations.
Minimalistic Physical Buttons
The Vision Pro headset features only two physical buttons on its body.
On the right side, there is a pressable knob, similar as Digital Crown, resembling the one found on Apple Watches.
A short press activates the main interface, while double pressing forces an exit. Long-pressing adjusts the interpupillary distance automatically. The interpupillary distance adjustment is the most elegant and painless I have ever experienced.
After a long press, the Vision Pro automatically detects my interpupillary distance and adjusts the lenses accordingly, completing the process in just 2 seconds. Even my exceptionally narrow interpupillary distance was perfectly calibrated.
The other physical button is the camera button, which Apple currently does not allow us to use yet.
Adjustments and Setting Up Eye and Hand Tracking
The tightness of the headset was adjusted using the large knob at the back right.
At this point, the setup of eye tracking and hand tracking was needed.
Eye tracking involved focusing on a small dot on the screen and rotating the gaze around it, both in a dark and a well-lit environment. On the other hand, hand tracking was simpler, requiring only a hand gesture.
After completing the setup, the real room captured by the headset’s cameras could be seen, with the familiar “Hello” message that Apple devices display when starting up.
In the actual experience, the eye tracking and gesture recognition of the Vision Pro felt seamlessly integrated. Wherever the eyes looked, objects were highlighted, and confirmation or dragging could be done with the fingers. All interactions were instantaneous, with no noticeable delay.
The eye tracking was extremely accurate, even for selecting the very thin options at the bottom, which could be instantly targeted with just a glance. There was no need to move the head; a quick movement of the eyes was enough.
For example, the browser’s address bar could be directly looked at, and it would be highlighted.
As soon as speaking was initiated, the Vision Pro would initiate voice input, which felt quite magical. There was no need to hold hands in front; one could comfortably sit on the sofa with hands naturally resting on the legs.
A gentle touch or light tap was all it took to give commands. Both left and right hands could be used.
It was the most elegant and intuitive VR interaction experience to had ever encountered.
The Most Elegant and Intuitive VR Interaction Experience Ever Had
The gesture and eye tracking features of Vision Pro are not groundbreaking innovations, as both PS VR2 and Quest Pro also have eye tracking, and devices like Quest and Pico have gesture tracking.
However, Apple’s brilliance lies in making these seemingly gimmicky features feel natural and incredibly user-friendly. It made me realize that hands-free control should be this intuitive and smooth.
App Experience
Next, let’s talk about the app experience. Even familiar applications can feel completely different on Vision Pro. Starting with the photo album, I saw how regular photos and panoramic photos transformed on Vision Pro.
By pinching the bottom control bar and moving it, I could freely zoom in and out.
Panoramic photos became curved screens that surrounded me in a 180-degree view, making them more enjoyable to appreciate than on a smartphone. However, I don’t personally take many panoramic photos.
AR Experience vs Quest
As for the AR display effects, Vision Pro has raised the bar significantly. It’s a hundred times better than the best AR experience I had previously.
Most VR headsets on the market capture external images using surface cameras and display them on screens. Quest 2 has black and white visuals, while Quest Pro offers color.
However, their effects are not impressive. The stitched-together images suffer from shaking, overexposure, and severe noise.
In contrast, Vision Pro utilizes multiple cameras on its surface, combined with the R1 chip, resulting in excellent color and clarity.
There was a moment when I couldn’t even distinguish between reality and virtuality.
Impressive Visual Effects and VR/AR Transition
There’s a moment when I adjust the brightness of the phone to the maximum, the screen does not get overexposed. There are only slight noise and some compromises in terms of FOV (field of view). I believe Apple has achieved the visual effects they promised at the launch event. It’s very stunning.
Built-in VR scenes
There is a built-in set of VR scenes in Vision Pro. By rotating a knob, you can adjust the degree of intrusion of VR scenes into reality, similar to opening and closing curtains.
You can switch between AR and VR.
Apple has an interesting design here. When we are fully immersed in VR and if someone’s gaze aligns with ours for about 1-2 seconds, that person will penetrate the VR scene and gradually appear in front of us.
If we turn our heads away and don’t look at them, they will slowly disappear behind the VR scene. The edges of the characters are smooth with gradient transparency, which is very elegant.
Exceptional sound experience and spatial audio
Vision Pro has the best sound experience I have ever encountered in a headset. The speakers have no compromises and support spatial audio.
When using Facetime, if we move the window around in front, behind, left, or right, the sound of the other person’s voice will come from that specific point, which is intuitive.
Facetime with a futuristic solution for displaying your face
When it comes to using a headset for Facetime, a significant issue is how the other person sees your face. Apple has come up with a somewhat sci-fi solution.
They use sensors to scan and create a 3D facial model, then use machine learning to simulate facial animations.
In actual experience, the effect of this persona is somewhat reminiscent of an early 3D game NPC, and it’s a bit funny.
Continue to Read Part 2 of 2 here