Immersive media has the potential to disrupt the journalism landscape in profound new ways. While technical issues still present limits, ambitious journalists are using all available platforms to get a glimpse of the storytelling media of the future. As immersive media increases in reach and functionality, 3D, AR, and VR platforms continue to further define how journalists will share their ideas. The most significant divergence from traditional media is the introduction of user-directed spatial dynamics, adding a new level of presence to readers. This will bring the concept of “spatial journalism” to the forefront of the industry.
While a video camera can tell a gripping story, it cannot put you in a space with agency. That will change. Until now, spatial journalism has been constrained by technical platform limitations, long production schedules, and relatively high costs compared to traditional media. However, technical advances are increasingly allowing these new stories to be delivered on more accessible platforms and with rapid delivery times.
What will spatial journalism be like?
It’s important to understand that this evolution of journalism is more than just using “cool holograms” to tell a story. Instead, it may use 3D imagery to immerse you, putting you in a cinematographer’s shoes. Imagine walking through a war zone, flying over 3D terrain maps, analyzing interactive data sets, standing next to Mick Jagger on stage, or bumping along in a rover on Mars. Commercial television had a profound and revolutionary impact on culture and society, but it can’t give the viewer agency within an experience. With augmented and virtual reality technologies being advanced by the likes of Google, Apple, Facebook, and Snapchat, users will soon be exposed to more intimate experiences. This heightened journalistic medium will be incredibly important for affecting empathy and etching storytelling elements into a user’s memory.
Joshua Foer’s book Moonwalking with Einstein illustrates how memory is highly dependent on spatial understanding. Before written stories were viable, the ancients surmised that spatial learning was better for memory, so they created “Memory Palaces.” For example, go to a dinner party and (if you’re like me) you will almost never remember the names of new acquaintances, but a week later you could likely draw an accurate diagram of the hosts’ home. If you can mentally place elements you wish to remember in that space, the memory will be more vivid and long lasting. The same can apply to impressions gained from quality journalistic storytelling. The effect of spatial journalism could usher in a new era of highly impactful stories that make a more lasting mark.
Technological challenges and advancements
Today, creating these experiences takes time. There’s creative technical challenges and distribution limitations to consider. So most spatial journalism focuses on feature stories, such as the mission to Mars by The New York Times or the rain forest fire from Time. These pieces can be considered learning and testing tools for their respective outlets, which have been advancing spatial journalism while pushing the limits of the platforms at the time. They are using every new technological advancement to probe users and judge interaction. You can trace these advances in immersive technology and how media outlets have tested spatial journalism within them.
Before 2017, these experiences could only be created within a specially downloadable app — a huge limitation on the spread of mixed reality (XR). For example, in late 2015, The New York Times had to give out around a million Google Cardboards and launch a new VR app just to show an immersive 360 video. Driving downloads is tough, and so is giving out a million Cardboards. Then the social media companies like Snap and Facebook (now Instagram as well) built AR platforms that media companies could piggyback onto.
So in 2017, when The Economist ran its Future of Food article, it used Snapchat and Snap codes to showcase AR on a wide scale with no new app. In 2018, Apple released Quick Look, which let you launch 3D objects in AR directly out of the browser. The Wall Street Journal followed with a “How To” on carving a turkey for Thanksgiving, but only for newer Apple phones. In each case, a new advancement was followed by an outlet experimenting with the new capabilities.
The technology behind these platforms is moving super fast, though, allowing for high rates of widespread adoption. Outlets like The New York Times are using webGL to put 3D experiences directly onto the mobile web. The NYT won an Emmy for showing a destroyed Syrian rooftop — an example of both a time-sensitive story and cutting edge technology. As seen in other examples of emerging tech, the time to market of this type of journalism will shrink and shrink as the technology around it expands.
What to watch for
There will also be democratization of spatial journalism. All of the media outlets mentioned are large international news sources. Using the old metaphor of Moore’s Law, we can see how this will get cheaper and faster, allowing regional sources and even straight clickbait to use these this new trend.
The technologies to watch for that will affect spatial journalism even further are 5G, the point cloud, and wearable devices. The impact of each of these cannot be understated. 5G will allow massive data loads to be dropped almost instantaneously onto devices. Imagine a journalist in the field taking live 360 imagery that a user can immerse themselves in – as if they are in the journalists’ spot.
The point cloud is basically a data mapping of the world which will allow shared experiences and object permanence in the XR world. Imagine walking by a coffee shop in NYC and getting a notification that it was once the infamous punk venue CBGB’s. At the time (in the future), The New York Times is running an immersive location-based story on the Ramones — so throw on your glasses and watch a Ramones concert in the space that made them famous; your friends can watch along with you too. This experience assumes wearable XR technology of course.
It’s hard to overstate the potential impact of wearables. Readers will soon be in the news, in many, almost unimaginable, ways. The first generations of wearables will be glasses with simple displays, like North, which will bring news into the field of vision, but not transform the experience. Once the hardware makers create glasses that access the previously mentioned point cloud, or at least depth sensing cameras to accurately map your world, a new age of spatial journalism and information will emerge. The most powerful sense, vision, will no longer be a shared experience; it will become personalized based off the 3D media the wearer (or maybe a bot) selects.
Maybe war won’t seem so noble once journalists can put you into the battlefield — where your whole field of vision and sense of agency are engrossed with a similar dread to those actually there. Or maybe you’ll better understand the weather forecast when you can plan your day not based off three icon raindrops, but an encompassing media production in your local sky. The worry is that wearables fracture the population from any semblance of a shared set of facts, like Neal Stephenson presages in his new book, Fall or Dodge in Hell. Imagine how different Facebook feeds are, and now project that dissonance onto the visual experience of the built world. But like most technologies, immersive XR will likely bring the good and the bad. That will be true for wearables’ effect on journalism as well.
There are current technological limitations that prevent saturation of spatial journalism. But large outlets are fully aware of its power and test their market with each new innovation. Spatial journalism will only make news and journalism more impactful as your memory and empathy are better served by spatially-centered experiences.
Mike Cadoux is the General Manager of QReal (formerly Kabaq.io), a 3D and augmented reality content creation and distribution company that usher’s brands into the immersive media world.
