Sony is actively working on powerful new PlayStation VR livestreaming and mixed reality features that might become accessible to users in the future — but they could be held back due to the PlayStation 4’s limited computing power. RoadtoVR reports that Sony R&D senior team lead Richard Forster disclosed the company’s latest VR experiments at a GDC session last month, though details are only now coming to light.
The good news: Sony has tested an enhanced VR version of the PS4’s existing Share feature, including several additions that would enhance livestreaming of VR content. First, the player could choose the position of a virtual third-person camera within the game world. Second, the first-person view could be modified for a fullscreen TV rather than displaying as cropped. And third, steadicam and filter effects could be applied to the output.
Since the additional camera angle might let viewers focus more on the player’s avatar than would normally be the case, Forster said that developers would need to tweak the way the player is rendered externally, adding extra kinematics that focus on the player’s own looks and movement. For instance, a full body — including a head with fake eye blinking and object tracking — would make the player look more natural to viewers than a disembodied pair of arms floating through space.
Another feature in testing would enable a “producer on the couch” to control the livestream output, making realtime selections and adjustments for the feed’s camera views, as well as possibly adding live commentary during the broadcast. A more sophisticated mixed reality version with an added real-world camera would provide an external view of the player, digitally composited within the video to appear “inside” the game — with separate foreground and background objects properly rendered. Sony used an Air Hockey green screen demo to illustrate how this feature would work.
The bad news: Between the processor demands of digital compositing and the current state of Sony’s software initiatives, these new features aren’t yet on the official PSVR SDK roadmap, and in some cases could be beyond the capabilities of even PS4 Pro hardware — which rightfully focuses mostly on producing fluid experiences for gamers. Forster said that one solution would be to use a second PS4 Pro for realtime broadcasting and compositing, which would clearly have enough extra horsepower to produce 4K and HDR output superior to what current PSVR video looks like.
While regular consumers would never consider dropping the cash on a second PS4 Pro just to share the output from a current-generation PSVR headset, professional livestreamers might. Sony also feels that the features could work at trade shows, help developers promote their games in livestreams, and assist in producing VR trailers that are more compelling to gamers. Whether they wind up in an official PS4 software update remains to be seen.
