Full-body tracking was once the stuff of movie studios and theme parks, but thanks to the proliferation of low-cost motion sensors, gyroscopes, and accelerometers, it’s finally beginning to trickle down to the consumer space. Trackstrap sells a product that works with the HTC Vive virtual reality (VR) headset, as does Perception Neuron. And for the most part, both capture joint and muscle movement just as well as their Hollywood counterparts.
The trouble is, they’re designed with stationary setups in mind. You’re pretty much out of luck if your VR or AR platform of choice happens to be mobile-first, like Google’s Daydream View, Samsung’s Gear VR, or Facebook’s Oculus Go.
That motivated Harsha Kikkeri, a former Microsoft engineer who helped to develop the Seattle company’s motion-tracking Kinect sensor, to build something a bit more flexible. His solution is the HoloSuit, a patented full-body tracker consisting of a jacket, a pair of gloves, and pants embedded with sensors that track movements, haptic motors that provide force feedback, and programmable buttons that correspond to computer, smartphone, wearable, and tablet controls.
Kikkeri calls it “motion tracking for the real world.”
“VR and AR are becoming more immersive and natural as resolution and field-of-view increase while latency decreases,” Kikkeri said. “We designed HoloSuit from the ground up to provide an unparalleled immersive, full-body and bidirectional AR/VR/MR experience.”
The HoloSuit is available for purchase on Kickstarter, and won’t begin shipping until later this year. But I got a sneak preview of it in action this month in New York City.
Body tracking
As I stepped into the wood-paneled conference room that’d been booked for my demo, Kikkeri greeted me enthusiastically. He wasn’t hard to spot — two motion-tracking base stations flanked him on either side, and a gaming laptop sat on a table nearby.
He was wearing the HoloSuit Pro — the pricier of the two available models — which packs a total of 36 embedded motion sensors (9 on the jacket, 22 on the glove, and 7 on the pant), 9 haptic motors, and 6 buttons. (The cheaper version has 10 fewer motion sensors.) It feeds a constant stream of angle, acceleration, and force data to a PC or mobile device wirelessly, via either a Bluetooth or Wi-Fi connection — the HoloSuit is compatible with Android, iOS, and Windows, Kikkeri told me, with OS X support soon to come.
Altogether, it took five years to develop the hardware, software development kit, and plugins for popular game engines including Unity and Unreal Engine, according to Kikkeri. And almost every element of the HoloSuit was designed and fabricated in-house, including the jacket’s textile work.
Above: Harsha Kikkeri demonstrates the HoloSuit’s flexibility.
Kikkeri, not one to waste time, kicked things off with one of his favorite showcases: a third-party digital avatar viewer. As he moved his right hand slowly upward, so too did a digital doppelganger on his PC. And as he turned from side to side, the avatar rotated on its axis.
I was struck by the HoloSuit’s low latency — Kikkeri’s movements were reflected on the PC nearly instantaneously — and equally impressed by the fidelity. Even subtle motions like a flick of the wrist or shrug of a shoulder didn’t escape the sensors’ notice.
That fine degree of precision makes the HoloSuit ideal for VR, Kikkeri said, because it’s incredibly immersive. The first of several experiences he’s prototyping is a submarine simulator that tasks players with operating knobs, buttons, and levers in a cramped control room. The suit’s buttons come in handy here: During the demo, Kikkeri used one stitched into the glove as a selection tool. (On Windows machines, it also serves as a left mouse button — the HoloSuit team is working on generic input drivers.)
The HoloSuit can also control objects in the real world, such as humanoid robots:
https://www.youtube.com/watch?v=sqoM25YqKVA
Submersible missile launches and robots aren’t all the HoloSuit is good for, though.
Next on the demo agenda was a cricket simulator created in Unity. (Cricket is one of the most popular sports in India, where Kaaya Tech — the startup behind the HoloSuit — is headquartered.) Kikkeri handed me the reins on this one; as I put on the jacket, which fit comfortably over my button-up dress shirt, he tightened stitched-in Velcro straps around my wrists to ensure the tracking was as accurate as possible. Then I — or rather, my digital avatar — stepped up to the plate.
My high school baseball days are long behind me, and it showed — the cricket ball whizzed past my avatar’s head more times than I could count. It wasn’t the HoloSuit’s fault, mind you — it tracked my flailing arms just fine, and its haptic motors painstakingly drilled home every swing and miss.
Kikkeri said the demo inspired a former cricket coach to create a virtual training program. I can relate.
I let Kikkeri demo the next two titles, HoloSuit Golf and HoloSuit Yoga. The former analyzes the golf strokes of suit wearers — there’s even a nifty cloud feature that lets HoloSuit wearers back up saved data to the cloud and share it with other people to view as a 360-degree video. And the latter, as you might expect, teaches complicated poses to amateurs like me step-by-step. (The HoloSuit’s haptic feedback motors vibrate when the wearer makes a mistake.)
HoloSuits everywhere
Kikkeri believes that the HoloSuit’s potential extends far beyond video games and virtual reality, and to that end, Kaaya Tech is exploring applications in health care, factory training, and education. Already, it’s partnered with a rehab clinic in Bangalore for digital training; a Bollywood music director to create a virtual orchestra and unique museum experiences in Romania; a university in Vietnam for surgery training; and India’s Defense Research and Development Organization on a disaster response program.
In the future, Kikkeri thinks it could be used to monitor patients in a hospital or to teach students about robotics. And that’s just the tip of the iceberg — his team is creating a software development kit that merges body movement data from the HoloSuit with facial movements captured via camera, and an augmented reality app that works with Microsoft’s HoloLens.
Above: One of the gloves that comes with the HoloSuit package.
Kaaya Tech’s plans are nothing if not ambitious, but the HoloSuit’s technology doesn’t come cheap. The less expensive of the two HoloSuit kits starts at $1,000; the premium model costs a whopping $1,600.
Kikkeri notes that some of its competitors, like the Teslasuit, are four to five times pricier, and that the HoloSuit works across all VR and AR platforms. No matter how you slice it, though, the price tag’s a tough pill to swallow.
But Kaaya Tech’s playing the long game. Kikkeri pointed to a recent study projecting that the AR and VR market will hit $162 billion in 2020, up from $5.2 billion in 2016.
“[T]he controller remains the Achilles’ heel of mixed reality, with cumbersome and unnatural handheld devices only tracking your hand movement in a limited range,” he said. “HoloSuit goes well beyond gaming and is so powerful that it can be utilized for real-life, mission-critical situations such as education, neuro-rehab, combat training, surgery, and even filmmaking.”
