On the 1-10 spectrum of “game developers should be able to sell completely broken demos” to “every game should be thoroughly vetted before release to earn the Nintendo seal of quality,” I’m somewhere in the middle. For years, I’ve hated how app stores and “games as a service” have cultivated a perpetually-in-beta culture for software, but as long as a game’s labeled Early Access or otherwise specifically disclaimed as unfinished, I can grudgingly accept that most games aren’t “finished” when they ship… except for titles that arrive without necessary features, such as Street Fighter V.
VR games are different. When a VR game breaks, the entire immersive experience can collapse, leaving you stuck inside a mess. Over the past two weeks, I’ve played “finished” games (most notably Killing Floor: Incursion and Ark Park) where objects become visually jittery to the point of inducing nausea, controls stop responding properly, and levels lock up, requiring a hard reset. These sorts of problems won’t surprise long-time VR users, but they can be show-stoppers for VR newcomers and demo players considering their first dip into this immersive tech. As Wipeout Omega Collection and Beat Saber have recently demonstrated, polished games can win even skeptical people over to VR as a platform. Unfinished, unpolished games can literally scare people away from VR forever.
My take: It’s time for VR developers, publishers, and retailers to stop selling half-baked games. The question is whether such a cultural change would best be accomplished by VR platform makers such as Sony, retailers like Steam, or the developers themselves.
Larger publishers, such as Ubisoft, could likely be trusted to ensure that their VR games (like Star Trek: Bridge Crew) have no experience-killing bugs. But small- and medium-sized developers need help.
The case for VR platform maker enforcement
In the 1980’s, Nintendo developed a Seal of Quality as a direct response to the industry collapse that Atari’s lax standards caused. The NES only played officially licensed games containing a Nintendo chip. The publisher tested, approved, and manufactured literally every game from every developer — until its authentication system was hacked. Even then, the chip succeeded in keeping most trash off the NES platform.
There were some serious problems with an all-powerful platform maker. Nintendo decided that it was going to be the Disney of games and censor certain content across both first- and third-party games. The company also set strict limits on how many games each developer could release per year, and it also made crazy, often dilatory choices regarding cartridge manufacturing and distribution.
But Nintendo got good results. Its focus on quality resurrected an industry done in by a glut of bad software. And developers learned to give their games an extra coat of polish before pushing them on customers.
For the VR generation, Sony is in the best position to replicate Nintendo’s model — or improve upon it. Sony already controls the manufacture of PlayStation discs (which are likely a small part of the total sales of PlayStation VR games) as well as the exclusive digital store for PSVR software. Adding an extra quality assurance step to the publishing process would be comparatively easy for Sony, though admittedly easier said than done.
Sony could playtest PlayStation VR game before release to squash bugs — not to judge content. To do this, Sony might have to hire a lot of additional testers and commit to being the “bad guy” who says no to shipping developers’ messed up games. But unlike 1980s Nintendo, it wouldn’t be doing this to control the market; Sony would focus solely on ensuring VR quality. Consumers and developers would be better off.
More realistically, Sony could get involved solely with developers that don’t have in-house quality assurance (QA) and testing teams, leaving bigger companies such as EA, Ubisoft, and Activision to handle all testing on their own. That’s how things are done today, and for bigger developers, it works: There’s no need to run every game through two or three gauntlets. But at a minimum, every VR game deserves at least one full pass from a QA team, regardless of whether it’s internal or external.
Above: Tripwire’s Killing Floor: Incursion is interesting and almost photorealistic when it works, but when it breaks, the visuals and controls become glitchy and unwatchable.
Why retailers and (some) developers can’t be trusted for VR QA
VR platform makers are arguably in the best position to mandate quality because they control the assets developers use for marketing — the PlayStation VR, SteamVR, Vive, and Oculus logos, to name a few. If any game can use a platform’s label without proper testing or quality assurance, what value does that label really have?
Unfortunately, the appeal of easy money without any minimum quality responsibilities is too much for some companies. Retailers will often sell virtually any game that might make money; if a lot of complaints come in, they just stop selling it and move on. Digital retailers have it even easier. After putting up an ungodly number of new games each month, Steam doesn’t worry about whether these are fully tested or not. Instead, it just automatically refunds payments for games played for under two hours within 14 days of purchase.
Small- and medium-sized developers aren’t always in the best position to police their own bugs, either. Many developers have become accustomed to shipping unfinished software, collecting cash from users, and fixing (or not fixing) the issues later. No Man’s Sky was perhaps the highest-profile example of this, but unfinished apps and games now ship hundreds if not thousands of times every day across various platforms. A developer’s reputation can be damaged by broken games, but fixed (at least somewhat) by patches.
It’s my belief that rushed developers and look-the-other-way software retailers such as Apple, Google, and Valve got us into this constant beta mess. Expecting any of them to fix it without some oversight isn’t realistic.
Above: The VR portion of Sony’s Wipeout Omega Collection for PSVR was developed by a relatively small team, but thoroughly tested before release.
So, what’s actually going to happen?
Will HTC, Oculus, or Valve create a seal of quality for their VR titles? I’m not holding my breath. PC gaming has been a Wild West of bugs for so long that there may be no baseline expectation of quality for either new accessories or software dependent on those accessories. Valve appears to be working to create some uniformity of experience between the varied SteamVR input and output devices out there, but even then, it seems to be focused on guaranteeing controller and headset compatibility — not software quality.
The console world might be different. Sony already has experience overseeing first- and third-party software releases, and it’s clearly done its best to make PSVR easier for average people to use than PC VR headsets. Establishing the PSVR as the “safe, properly tested” VR option of choice — again, putting content-screening aside — could be a winning angle for Sony and PSVR gamers alike.
For the time being, I’m leaving out Microsoft. It doesn’t make VR hardware, but currently has a Windows Mixed Reality certification program for third-party devices, and an expensive, low-selling AR headset (HoloLens) of its own. A more tightly curated Mixed Reality store — and a plan to put VR on the Xbox One — could give Microsoft an edge over PSVR and SteamVR. But then, expecting Microsoft to police developers for bug-free software might be unrealistic given its own track record.
Gaming’s beta culture hasn’t been around forever. I’m hoping that the awful experiences caused by improperly tested virtual reality experiences will help to quash that culture in the VR gaming world, before it’s too late.
