How Adobe used its huge data bank to build Sensei, an AI tool for creatives

There’s mounting competition over AI and bots among the tech giants. Amazon touts Alexa, Google has its Assistant, Microsoft has Cortana. But at Adobe, an often overlooked player in this contest, it’s all about Sensei.

Launched last fall, Sensei is a series of AI services and a voice-powered virtual assistant being added to Creative Cloud (formerly Creative Suite) apps and services like Photoshop and Premiere.

Some Sensei services are already available, like the ability to change a facial expression with Face Aware Editing in Photoshop, while others, like the ability to control Photoshop with your voice, are still prototypes.

Sensei will be able to talk you through how to edit photos and videos like a pro because Adobe has tracked millions of photo and video editing sessions. Sensei AI will power emerging Adobe tech like painting in VR with Project Dali, Adobe’s answer to VR painting apps like Quill from Oculus; and the photo restyling tool Artistic Eye, Adobe’s answer to apps like Prisma and Artisto.

Data from trillions of transactions recorded by Adobe Analytics and other Marketing Cloud services will also be harnessed to provide intelligent services.

In an interview with VentureBeat, Adobe CTO Abhay Parasnis says the company does not want to build a generalized AI like Alexa or Assistant. Instead, Adobe is aiming to build a specialized intelligent assistant with a variety of AI services built for creatives.

Today Adobe uses computer vision for things like auto-tagging sunsets or manipulating photos, but soon it will extend computer vision services to businesses and developers who want to analyze images or give their products sight, Parasnis said. 

With Adobe preparing to announce more Sensei features next week in Las Vegas at its annual Adobe Summit conference, we sat down with Parasnis to learn more about Adobe’s Al ambitions.

This interview has been edited for brevity and clarity.

VentureBeat: Google spends a lot of time talking about how search is part of its specialized AI. Microsoft has business. Is Adobe computer vision?

Parasnis: I would say more broadly if you ask me in that sense it’s the creative domain of which imaging, video, computer vision, then the marketing PDF domains. … Computer vision — I look at that as a subset of the world created because we already make recognition, video analysis, image tagging … all things we’re very focused on.

We were shipping a lot of these machine learning services well before we announced Sensei as kind of the unified approach. Like Content-Aware Fill services we were shipping well before we ever talked about Sensei. So our mindset has been, let’s build user-facing features of AI first before we just say AI for the sake of it. Like it has to be something in the service of the experience.

There are players in the market that are going after what I would call general purpose AI, this notion of very broad, general-purpose platforms for AI that can pretty much solve any problem you throw at it.

We look at that and say that is not our play. What we are trying to do with Adobe Sensei is the other end of the spectrum, which is deeply domain-specialized AI. There are three domains where we think we have very unique expertise, content, and data assets at scale that we can really use to train and learn these models.

Not surprising we’re focusing on the future of creativity and creative AI and the notion of deeply transforming documents with intelligence in it.

VB: Time and again I hear the mantra that AI is only as smart as the data you have. Can we dive into the data? What are the datasets behind all this?

Parasnis: We do believe that AI is not just about algorithms. It’s actually dependent a lot on the datasets and content you feed through to make that. So first we agree with that.

So take the example of the Creative Cloud. We have literally hundreds of millions of images, but these images are not the cat pictures I’m posting or something. These are very highly curated, highly produced images, so they are very high quality datasets tools you can use to train.

Second, we have years worth of anonymized instrument data on which tools people use the most. Let’s say you’re trying to change the lighting effect on a face. We can see that these four tools get often used by pros who are really good at Photoshop; they always use those four tools to get the intended effect that is actually at the pro level of publications. There’s a notion of what is the best lighting effect because we have so much data that we can basically have the model trained to do that automatically. That’s on the creative side.

The Marketing Cloud side is another example where we have massive amounts of data. Adobe Analytics processes something like 90 trillion transactions now a year, so that’s another example where the AI system can get trained so much better because the coverage of the dataset is very large.

Sensei for us is a common approach in platform and framework with all three domains — creative, document, and customer analytics — because at the end of the day, customers use all these tools in concert. They use Creative Cloud to create the content, then they use Marketing Cloud to deliver the content and Analytics to measure how it’s performing. Having the full journey allows us to do a Sensei stack that’s very unique because it goes all the way.

VB: What’s the timeline for Sensei’s rollout?

Parasnis: Next week in Las Vegas we’ll roll out some pretty big new things in Sensei. But think of this as already live and being used by a lot of customers today, and we’ll just keep adding more and more features.

VB: What’s the educational potential for this? Would Sensei help somebody who wants to be more than a novice photographer?

Parasnis: Yeah, we kind of look at that as democratizing the skills set, absolutely. We think this has a significant potential from a learning, training, skills enhancement perspective because today it takes a lot of time for people to kind of get mastery of these complex tools. They’re very powerful, but they take a long time to master. And what AI will do in particular is take that learning curve dramatically down so a lot more people can get there.

VB: With all the datasets you have available, are there applications for commercial partners or businesses? Are there industry examples where Sensei may provide computer vision services?

Parasnis: The other half of the Sensei story is that we have talked about externally a little bit, but in fact that’s one of the things I’m going to be onstage at Summit talking more about, is that Sensei is a platform for third parties and ISPs and developers.

We are exposing it as an API platform where you can bring your own dataset in and apply our services against your own — and not just computer vision, but more broadly.

Adobe I/O is our developer platform and kind of how third parties connect with our dataset. Now I will say there are a lot of customers who already connect with Adobe analytics and use that data, so we are very much focused on Sensei as a platform for others to plug in and we will let them do what you’re describing. I will say broadly, as we build all these services, we look at what makes sense for us to put in our own apps and then what makes sense to open up for APIs to kind of do that — image recognition, computer vision, all those will be good candidates.

VB: Thank you.