When Facebook’s Oculus unit revealed a next-generation prototype VR headset called Half Dome in May, one of its signature features was a “varifocal” display system that actually moved the screens inside to mimic focus changes in the user’s eyes. Now the company has revealed DeepFocus, the software rendering system used to power the varifocus feature, along with an interesting announcement: DeepFocus’ code and data are being open-sourced so researchers can learn how it works, and possibly improve it.
While the details of DeepFocus are technical enough to be of interest only to developers, the upshot is that Oculus’ software uses AI to create a “defocus effect,” making one part of an image appear much sharper than the rest. The company says the effect is necessary to enable all-day immersion in VR, as selective focusing reduces unnecessary eye strain and visual fatigue.
Oculus says that DeepFocus is the first software capable of generating the effect in real time, providing a realistic, gaze-tracking blurring of everything the user isn’t currently focusing on. The key word there is “realistic,” as prior blurring systems developed for games have been developed for flashiness or cinematographic effects, not accuracy, and Half Dome’s mission “is to faithfully reproduce the way light falls on the human retina.”
To accomplish this, the company paired deep learning AI tools with basic color and depth RGB-D data already provided by game engines. A DeepFocus development team created 196,000 images randomly filled with objects, training the system to properly render blur in scenes with even greater complexity and variability than typical VR environments.
“With deep learning, our system was able to grasp complex effects and relations, such as foreground and background defocusing,” explained Oculus research scientist Anton Kaplanyan, “as well as correct blur at occlusion boundaries. And by generating a rich database of ground truth examples, we were able to cover a much wider range of defocus effects and set a new bar for depth-of-field synthesis.”
While DeepFocus worked smoothly on 1080p graphics, the team later upscaled its work to handle the demanding image-rendering pipelines of VR devices, ultimately introducing a four-GPU pipeline to process modern VR-quality imagery in real time. At the moment, the team is still working to get DeepFocus running at high VR resolutions on a single GPU, a process that’s still underway.
Facebook’s stated goal in open-sourcing DeepFocus is to share its knowledge with VR system developers, vision scientists, and perceptual researchers; clearly, the information will help software developers prepare future VR apps, as well. But the timeline for actual release of a VR headset with varifocal hardware is unclear. Oculus says that it developed the defocusing software for Half Dome, but sees it as hardware agnostic and believes that it will be useful in the next generation of VR headsets.
Reading between the lines, that would suggest Oculus is not ready to bring a varifocal solution to market — a conclusion that would match the company’s reported decision to make a “Rift S” VR headset that’s iterative rather than a big step forward, focusing more on convenience than major new technologies. That controversial decision would make more sense in light of today’s announcement, as virtually no one would own the quadruple-GPU PC Oculus says is currently required for rendering varifocal imagery. Giving developers an opportunity to speed up the rendering software might help bring truly next-generation VR headsets to market faster.
