testsetset
Google today is releasing a working interactive version of AI Duet, an app that lets you play a virtual piano with accompaniment from a computer system that’s designed to riff on what you play.
Yotam Mann, a musician working at the Google Creative Lab in New York, first unveiled the app at a company event in San Francisco in November. Around that time, Mann also open-sourced the code. But it takes some effort to get the code running on your own. Now Google is making the app available on the web. If you have a PC in front of you, you can use your keyboard to play the virtual piano’s keys, and if you’re in front of a mobile device, you can just tap on the display and get a similar effect.
It’s very cool — so cool that it won the “best demo” award at the 31st annual Neural Information Processing Systems (NIPS) conference in Barcelona in December.
To do its thing, AI Duet relies on Tone.js, an open-source web audio library that Mann created. But to know how to respond to user input, AI Duet also depends on Magenta, a library that’s built on top of the Google-led TensorFlow open-source framework for deep learning, a type of artificial intelligence (AI) that generally involves training artificial neural networks on data and then getting them to make inferences about new data.
June 5th: The AI Audit in NYC
Join us next week in NYC to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.
Like Facebook, Microsoft, and other companies, Google is using deep learning in more and more of its services — from Android Wear to Google Translate. Google wants people to understand what the underlying deep learning software can do. The Magenta initiative, which is designed to showcase art and music generation with machine learning, fits in well with that goal.
“Someone like my mom, who’s not technically savvy — she can actually play with this thing,” Doug Eck, a Google research scientist working on Magenta, told VentureBeat in an interview. “I can’t wait to show this to her. That’s seriously an important outcome for AI Duet — to understand what these models are able to do.”
Mann also contributed to other so-called AI Experiments: Bird Sounds, Giorgio Cam, the Infinite Drum Machine. I’ve seen them all, but personally, I’m deeply fascinated by AI Duet. It celebrates human creativity and plays with it.
From time to time, AI Duet will respond to your playing with one or more notes that might seem surprising — maybe higher or lower than what you played, maybe longer or shorter. This is a good thing and by design. “I think surprise is one of the most important things you can talk about if you’re doing generative media,” said Eck, who spent his early 20s as a musician before pursuing a Ph.D. in AI and music at the University of Indiana. “If everything is predictable, it’s just not that interesting.”
Sure, this is just an initial implementation, and Google could improve it in any number of ways. For instance, Google could make it possible for two or more people to play alongside Magenta, or refine the way Magenta performs, or let people donate their performances with Magenta to make the software smarter. But what’s out now is a significant start. At the highest level, it’s a fine demonstration of how AI that exists today can make content that’s neither entirely predictable nor totally crazy.
