Unexpectedly announced at an early CES 2019 media event, HTC’s latest and highest-end VR headset is the Vive Pro Eye — an upgraded version of the already premium Vive Pro with integrated eye-tracking hardware. The eye tracking can be leveraged for in-app controls, analysis of user attention during training sessions, and foveated rendering.
If you’re not already familiar with foveated rendering, it’s about to be a big deal for VR. Cameras inside a headset precisely and quickly track the position of your pupils, enabling the GPU to know where it needs to focus its rendering resources — and where it can skimp. One Vive Pro Eye developer said that with foveated rendering the GPU was saving 30 percent of its power over standard rendering, performance that can be saved to conserve energy or used to increase detail within the area viewed by the pupil.
The technology would be ideal for high-resolution gaming, but Vive Pro Eye is specifically being marketed at enterprise customers, and HTC suggests the still-unconfirmed price will be another step up from Vive Pro. Most gamers aren’t willing to pay extra for the standard Vive Pro, so game developers won’t likely make eye-tracking games for the Vive Pro Eye.
One key selling point of the headset is a rapid-fire setup process that lets users enjoy a powerful VR experience with little manual adjustment — nothing more than turning an IPD knob to properly align the displays horizontally with your eyes. Ideally, this would be automated, but the headset uses blue dots to show you exactly where your eyes are located, and you just have to turn the dial to align two circles with the dots.
Above: The knob to adjust IPD is a tiny black dial on the edge of the headset, shown here on the left, but normally on the right when worn.
Once that’s done, Vive Pro Eye’s screens go grey, and a blue dot moves through five positions to see that your pupils are able to register center, northwest, northeast, southeast, and southwest positions. Cameras hide inside the unit’s dark, well-padded chassis to continuously monitor your gaze.
Above: A view into Vive Pro Eye’s interior, including gaze-tracking cameras.
HTC believes that the likely customers for Vive Pro Eye will be luxury retailers, businesses seeking new communications tools, professional athletes, and other enterprise users. During CES, the company offered a bunch of demos to show off the Vive Pro Eye’s capabilities. Here are a few examples.
ZeroLight BMW test drive
The only Vive Pro Eye demo to use both eye tracking and foveated rendering, this app allows the user to experience a retail showroom style VR walkaround of the BMW M5, plus the opportunity to sit down in the car and watch it take off at a racetrack.
Developer ZeroLight showed off the foveated rendering feature with two modes. Split-screen allowed viewers to compare “standard rendering,” where everything is rendered with the same level of detail, versus foveated, where only a portion of the screen was rendered in high detail and other parts were rendered with less detail.
The difference was nowhere near as apparent as when ZeroLight activated a green/red mode that used green to indicate the high-detail rendering circle where your eye is looking and red for everything else. Pixelization is much more clearly evident in this mode — just look at the red in the screenshot above — but the key is that when the feature is working normally on Vive Pro Eye, the average user gets the benefits without even noticing it’s there.
MLB Home Run Derby
This demo used eye tracking in a basic way, permitting gaze alone to enable or disable menu settings by flipping switches if you looked at them for a few seconds. While the feature sounds boring, it actually enables a headset wearer to completely do away with a traditional controller for the purposes of starting up a game and then grab a buttonless controller, such as a tracked baseball bat, and start swinging.
Home Run Derby worked well on the Vive Pro Eye, even though HTC doesn’t expect that the headset will be used by gamers. Company reps suggest that sports training apps will instead be used by professional athletes and coaches to make sure players are focusing and performing to the best of their ability.
Training, simulations, social VR, and more
Several other demos showed off potential uses of the Vive Pro Eye’s gaze-tracking feature. Ovation showed a public speaker training app that uses eye tracking to make sure you’re properly focusing on your audience, rather than on your teleprompter or notes, when addressing a crowd.
Lockheed Martin’s Prepar3D is a complex flight simulator, augmenting the user’s physical flight stick and throttle controls with gaze tracking that can be used to activate the numerous subsystems within a fighter jet for a drone shoot-down mission.
A business-focused social VR application enabled up to 20 people to co-exist in a virtual presentation space where objects, videos, and Powerpoint presentations could be easily called up for group viewing, sketching, and collaboration. Gaze tracking was used in an extremely basic way to make the eyes of 3D avatars actually show where people were looking during their sessions together, but in my demo, my collaborator’s eyes weren’t being tracked.
It’s presently unclear which company is providing the eye-tracking hardware, as Tobii has said it won the HTC supply deal, but an HTC press release clearly lists 7invensun as the provider. Numerous demos at the event were experiencing hiccups with the eye-tracking feature, requiring software reboots that appeared to have something to do with eye tracking-specific driver issues — but when it worked, it worked very well.
Regardless of where the new feature’s hardware comes from, HTC and its partner have plenty of time to debug the software between now and the second quarter of 2019, when Vive Pro Eye is expected to be released. I expect that the final version of the headset and its apps will work flawlessly with proper code.
