How Google’s Pixel 2 Now Playing song identification works

The most interesting Google Pixel 2 and Pixel 2 XL feature, to me, is Now Playing. If you’ve ever used Shazam or SoundHound, you probably understand the basics: The app uses your device’s microphone to capture an audio sample and creates an acoustic fingerprint to compare against a central song database. If a match is found, information such as the song title and artist are sent back to the user.

Now Playing achieves this with two important differentiators. First, Now Playing detects songs automatically without you explicitly asking — the feature works when your phone is locked and the information is displayed on the Pixel 2’s lock screen (you’ll eventually be able to ask Google Assistant what’s currently playing, but not yet). Secondly, it’s an on-device and local feature: Now Playing functions completely offline (we tested this, and indeed it works with mobile data and Wi-Fi turned off). No audio is ever sent to Google.

It’s worth noting that Now Playing is turned off by default. You have to explicitly turn it on in the setup flow when first starting your Pixel 2 or Pixel 2 XL, or in Settings (as shown above).

We asked Google to explain how the feature works.

Detection

Now Playing uses your device microphone to grab audio samples and machine learning to distinguish which parts are music that can be converted to a digital fingerprint. It then tries to match that fingerprint to the local song database. In optimum conditions, a Google spokesperson estimated that detection should take just a few seconds. If there’s background noise, however, it can take a little longer, we were told.

If you’re seeing a longer delay, and you likely will, that’s because Now Playing isn’t working all the time. To save battery life when you’re listening to continuous music, Now Playing only runs every 60 seconds, the Google spokesperson explained. That means if the last detection was 30 seconds ago, you’ll only get an update on the next song that is playing in another 30 seconds, plus the time it takes for the actual recognition. If after 60 seconds no music is playing, the system will wait for music to be detected before attempting a new song identification. This also explains why long after a song is over, your lockscreen will still show whatever was previously playing.

When a song is identified and displayed on your lockscreen, you have the option to tap on it. After you unlock your phone, you’ll be taken to the Google Assistant where you can learn more about that given song.

The feature is purposefully designed as an “ambient” and “lean back” experience as opposed to on-demand. My colleague Khari Johnson lamented as much in his Pixel 2 review — there’s no historical option or way to see all the songs Now Playing has detected as you go about your day. It’s a “currently playing” feature and that’s it.

Database

The Pixel 2’s on-device database for Now Playing is based on Google Play Music’s top songs, the Google spokesperson revealed. Google wouldn’t share the exact number of songs in the database, but the spokesperson did note it’s in the high 10s of thousands (for comparison’s sake, Shazam claims its database features over 11 million songs, but that of course is in the cloud).

If you’re curious, XDA posted over 10,000 songs that the Pixel 2s can detect. While it’s cool to sift through this list, you should know that it’s far from definitive, for multiple reasons.

First up, the Google spokesperson revealed that your device will have a different song database depending on the country you’re in — there’s a different catalog for every country where Pixel 2s are sold. Secondly, Google also confirmed that the database is updated weekly, depending on what’s popular on Google Play Music in your country. This weekly update, which only ever occurs over Wi-Fi, is incremental and replaces the most outdated part of your country-specific database with new songs.

Since Now Playing works automatically and Google made a point to keep it localized, these constraints created a new limitation: local storage. That’s why the feature doesn’t work across all songs in Google Play Music, but rather just the most popular ones (not necessarily most recent, just “top music”).

The Google spokesperson wouldn’t give us an exact size for the database file (which is not surprising, since it changes every week and is based on your country) but did say the whole feature should take up less than 500MB. Again, if you never turn the feature on, don’t worry — you won’t lose this space.

Exclusive

At the moment, Now Playing is exclusive to the Pixel 2 and Pixel 2 XL. There are no plans to bring it to other Android devices, not even Google’s own Pixel and Pixel XL.

The Google spokesperson noted that Now Playing requires specific hardware and software changes, but didn’t elaborate exactly what was required aside from saying that the two have to be closely integrated for the feature to work. It’s not out of the question that future devices, with the specific hardware and software changes, could offer Now Playing as well, but Google can’t simply bring the feature to Android Oreo as an update, we were told.

In fact, Google doesn’t view see Now Playing as an Android or even a Google Assistant feature. It’s very much a Pixel 2 feature, at least for now.

Now Playing is something I believe will lead to conversations, and I’d even go as far as betting it will be the most-discussed feature among friends of Pixel 2 and Pixel 2 XL owners. It’s really the perfect storm: Everyone has heard of Shazam and SoundHound, so nobody will feel out of their depth discussing the topic; nothing like Now Playing is available on other phones; and music is always a timeless topic.

For those reasons, I suspect Google will try to keep Now Playing exclusive to its Pixel 2s for as long as possible.