testsetset
During a session at Google’s I/O 2015 conference headlined by the Advanced Technologies and Projects Group (ATAP), engineers demoed what they called Project Soli, a novel gesture-recognition technology bound for handheld devices. The promise of the tech was that you could interact with things without actually touching them, which would ostensibly open up all manner of new ways of performing tasks.
After a little over four years in development, Soli emerged in the Pixel 4 series as the gesture-detecting Motion Sense. So was it worth the wait? We used the Pixel 4 for a week to put Motion Sense through its paces. While our impressions are on the whole quite positive, it’s clear Soli is a long way from fulfilling its potential.
Radar
At a high level, Soli uses a combination of high-speed sensors and data analysis techniques like Doppler to detect the size, location, and proximity of objects within its 180-degree field of view — through materials — with sub-millimeter accuracy. Electromagnetic waves at the 60GHz frequency reflect information back to an antenna, and data derived from the reflected signal (like time delay or frequency changes) provides Soli hardware information about the interaction.
The latest public information about Soli suggests it uses around 0.054 watts and can track up to 18,000 frames per second, which is granular enough to detect when someone rubs their thumb and forefinger together. In fact, the latest iteration of Soli can theoretically detect a person standing up to 23 feet away behind a wall.
June 5th: The AI Audit in NYC
Join us next week in NYC to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.
Above: The glow indicating that Motion Sense detects an object.
The Pixel 4 series implementation is a bit more conservative than that used in onstage demos to date, for better or worse. It’s capable of detecting just three classes of motions:
- Presence: When the Pixel 4 or Pixel 4 XL is laying flat on a table, Motion Sense detects objects between 0.6 inches and a foot away. In the absence of motion, it switches the phones’ display to a power-sipping ambient mode.
- Reach: Motion Sense detects when a hand makes a gesture toward it and quickly turns on the phone’s screen and facial recognition sensors (to expedite the unlocking process). Additionally, if there’s a blaring ringtone or alarm, they’ll be briefly silenced.
- Gestures: A hand wave from up to about a foot and a half away across the Soli sensor dismisses incoming calls and snoozes alarms. And if there’s a supported music app running, it skips to the next song in a playlist.
Across all three motion types, a sliver of the Pixel 4 series’ screen beneath the top bezel glows to indicate when the Soli sensor detects motion.
The reach gestures worked exceptionally well in our testing, consistently waking the Pixel 4 series from sleep and priming facial recognition for a speedy unlocking process. But we’d be remiss if we didn’t note that iPhones with Face ID achieve essentially the same effect with an accelerometer.
Soli’s gesture recognition is a better showcase of its robustness, but only by a smidge. Swiping right-to-left or left-to-right across the Soli sensor reliably advances to the next or previous song in a playlist within music apps like Google Play Music, Spotify, and Deezer, even when those apps are running in the background. And that same gesture snoozes (but not dismisses, unfortunately) alarms like a champ.
Software
Developers don’t have an enormous amount of freedom with Motion Sense at the moment, but they’re able to build in support for playback controls through Android’s media controls API. In addition to Google Play Music, Spotify, and Deezer, Motion Sense gestures work with Amazon Music, Apple Music, Pandora, Shazam, SiriusXM, and over 15 others.
Above: Switching songs with Motion Sense.
Motion Sense has a small but budding game ecosystem, too. Google teamed up with the Pokémon Company to create Pokémon Wave Hello, a promotion for the upcoming Pokémon Sword and Shield that serves as an introduction to Motion Sense’s basic gestures. Swiping a finger above the Pixel 4’s or Pixel 4 XL’s screen prompts a reaction from one of five selectable Pokémon, while an upward swipe reveals a dashboard with each one’s height, weight, type, and ability stats.
Above: Pokémon Wave Hello on the Pixel 4.
Another marquee Motion Sense experience is Headed South by Ustwo Studios, the development house behind the blockbuster series Monument Valley. In it, you use touch to nudge your character — a bird named Soli, a not-so-subtle nod to Project Soli — toward a non-player bird’s slipstream in order to catch up to it. A swipe across the Soli sensor provides a temporary boost in speed.
Above: Demoing Headed South.
Stretching the definition of “games,” the Pixel 4 series comes preloaded with a series of six “Come Alive” wallpapers that respond subtly to motions. In Prickly, a pastel cactus bends in the direction of your wave, and in Doodle abstractionist doodads extend and snap back elastically to their original positions.
Above: Demoing Motion Sense with one of six Come Alive backgrounds preloaded with the Pixel 4.
It’s early days, but the first crop of Motion Sense experiences has us intrigued. (Gesture-enabled Candy Crush or Fortnite, anyone?) Of course, there’s the matter of the Pixel 4 series’ install base, which might deter developers from investing outsize effort into Motion Sense games. But we’re excited to see what the future holds.
Limitations
Another potential adoption deterrent? Motion Sense isn’t supported in every region and country the Pixel 4 series ships to. That’s because Google has to certify the sensors with regulatory authorities in order to legally transmit at the required frequencies. In fact, to avoid interfering with navigation systems and military equipment, Motion Sense will disable itself when you enter unsupported regions or when airplane mode is enabled.
At launch, Motion Sense works in over 50 countries, including the U.S., Canada, France, Germany, Italy, Denmark, the Netherlands, Spain, Sweden, the U.K., and the United Arab Emirates, but absent from the list are Mexico, New Zealand, and others. Google reportedly wasn’t able to obtain a license from the Indian government to use Soli in the country and plans to launch the Pixel 4 series in Japan early next year without Motion Sense features.
Growing pains
Like any new technology, Motion Sense has a lot of maturing to do. It works as advertised today, but the locked-down nature of its APIs and the dearth of app support out of the gate (excepting media apps) makes evaluating its potential difficult.
Assuming Google delivers on its promise to evolve Soli in the coming months and years (which is by no means guaranteed), Motion Sense could become one of the most compelling reasons to buy the Pixel 4 or Pixel 4 XL. At present, however, it’s little more than an (admittedly useful) parlor trick.
