The release of Apple’s ARKit and Google’s ARCore has sparked a burst of creative energy. We are seeing a new wave of demos from developers all over the world and there is a growing possibility that a paradigm shift in computing could be just around the corner when AR takes over mobile as the dominant consumer platform.
As a veteran in the AR space, we’ve been waiting for this moment for a long time. After leading 90,000 hours of mixed reality development I can tell you we’ve learned the harsh reality of AR development first-hand. There is a long road between demo and a live project; the act of augmenting reality can take much longer than you might expect. Sometimes the challenges should have been obvious, but more often than not they come from the messy process of invention and applying innovation to the real world.
Although exciting, these early demos we’re seeing are likely to run into similar challenges that we too faced in the early days of building AR experiences. However, the results are worth it! Consumer engagement with AR provides companies with tremendous opportunities to improve products and tell brand stories.
Here are our top considerations from the frontlines of AR/MR that developers should take into account when building mixed reality experiences.
Above: From prototype to live AR app, courtesy of Trigger.
1. Hardware limitations
Both ARKit and ARCore target later release phones. So a large number of consumers will be left out until they upgrade their hardware. Neither Google nor Apple’s solutions are currently cross-platform, so developers will have to port from Apple to Android or visa versa to accommodate the most users possible.
2. You still need an app
ARKit and ARCore are both integrated into the OS of the device, which provides great performance advantages over third party software which sits a layer above. However, the code containing the content and the experience still requires an app download. Creating the app is just the tip of the iceberg – marketing the app and driving downloads are some of the most daunting challenges.
3. Nobody wants a one trick pony
To make an experience worth downloading, developers should avoid any singular experiences. You can have the best animation in the world, but it will grow tiresome after a single repeated viewing. Good questions to ask yourself are: what can the user do in the first second, the first minute, the first day and the first week, and make sure the experience evolves. The only way to solve this besides deepening the experience is to come up with a dynamic solution that’s different every time. The Machines is a good example of using multiplayer to make a game replayable even if the content runs out.
4. 3D is expensive
Above: 3D development in AR can be expensive
Budget is often the “dream killer” and much of that can go into creating 3D assets. Clients either don’t have 3D assets readily available or if available, they may not be in a usable format. There can be a lot of work involved in bringing an asset from film/VFX resolution or CAD manufacturing files, down to what will work on a consumer’s mobile phone. The amount of 3D required is also an issue. As mentioned before, variety of content is important. Sometimes though it can be an issue of completeness. For instance, it’s easy to demo placement of a single coffee table, but what happens when the brand has hundreds of products? The IKEA Place app is a great example of a large catalogue of 3D objects with over 2000 items available in AR!
5. Locations can be constraining
There’s a reason why so many ARKit demos are shown in wide open areas. Collision and occlusion are a huge issue because ARKit and ARCore can’t detect or account for this. So that same demo that looked great in a wide open space, suddenly looks very broken inside a coffee shop or classroom. There are many more considerations when location comes into play, which can vary greatly. We’ve had to work around fast food table sizes, ideal foot traffic in retailers, changing lighting conditions in theme parks and even magnetic interference around landmarks to name a few.
6. Avoiding Fatigue
Holding up a phone, which we call the “device up” experience can be surprisingly tiring. After years of prototyping and user-testing, we know that different user types have different conditions or thresholds for fatigue. For example, some young kids have trouble holding up a phone and tapping the screen simultaneously, while an adult out shopping will only have one free hand for their phone. Fatigue also affects hardware because AR requires an open camera which drains the battery very quickly and heats up the device.
7. Being Social
AR is most powerful when paired with other technology. Users naturally want to share their AR experience because it’s the only technology that places 3D content into their own world. In addition to selfies, Snapchat’s world lens uses AR to create customizable and shareable scenes. However, social is only the start: live data can make AR timely, GPS and beacons can make it location aware, and a variety of enterprise features can make AR practical and useful.
8. Consumer Messaging
This is probably the most neglected and difficult part of the equation. A consumer wants to know why an app is worth downloading and feel comfortable with how it works before they commit. Communicating this message can happen across different media and social platforms, and sometimes it ends up the responsibility of an employee in store. Once a user downloads the app, it is also important to provide instructions on how to use it and what to do if things appear not to work.
9 – Don’t forget the “physical” part of digital play
AR in an open space or on an empty table can be a very engaging experience, although we’ve found that AR is exponentially more powerful when it involves something physical under the digital layer. Placing a digital character in your world is fun, but it’s even more fun when you can walk that character up to a toy car, open the door and then drive off in it. A virtual pet running around on the grass is cute, but it becomes much more interesting when it looks for a real tennis ball, or knows to scratch at the door to go inside. In the Chalk AR App you can draw over the real world in AR with another person. That layer of interaction with the physical world can make all the difference in an AR app.
10 – This is just the beginning
Above: Jason Yim, CEO of Trigger, with future AR designers and engineers
Last year Pokemon Go introduced the world to AR. This year ARKit and ARCore will deliver the first true AR experience to many consumers. While existing AR platforms like Vuforia, are already integrating with ARCore and ARKit to enhance their features, these technologies are just scratching the surface and the challenge for developers will be to stay relevant as new advancements continue to emerge. Just when we think we’ve sorted out mobile AR, head-mounted AR will finally mature and shift the whole industry once again.
Jason Yim is the CEO and executive creative director of Trigger, The Mixed Reality Agency with more than 90,000 hours of development in AR/VR/XR.
