Watch all the Transform 2020 sessions on-demand here.
Stock photo giant Shutterstock has unveiled some nifty new search smarts that use computer vision to help you find images based on their composition.
The new “composition aware” search tool is available now in beta within Shutterstock Labs, and it promises to change the game in terms of unearthing the exact kind of photo layout you want for a project.
As with the standard image search tool, you enter keywords in the search box — for example, “man” and “woman.” But if you enter each word as its own search term (using a comma between the words), you’ll be able to drill down into the contents of the photo and search for the composition you’re looking for.
Down the left-hand side you’ll see a box with a little circle for each entity in your search terminology — in this case, “M” and “W” for “man” and “woman.” The default in this instance shows images where the man is on the left of the photo, and the women is on the right. Though some of the results don’t exactly match this criteria, it is broadly accurate.
June 5th: The AI Audit in NYC
Join us next week in NYC to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.
Now, if you drag the little circles around the box on the left, the image results change accordingly, so that you can see images where the woman is on the left and the man is on the right. Again, some of the results aren’t 100 percent reflective of the action here, but based on our tests it is pretty accurate.
Above: Woman & man
The same principle applies to other search terms, such as “wine” and “cheese.” You can drag things around so that the photos of “wine” are on the left…
Above: Wine & cheese
… or on the top right-hand side of the image.
Above: Wine & cheese
Another interesting facet to this new tool is that you can search for images that have space for text — for example, as part of an advertising campaign in which you wish to overlay some marketing blurb on a photo.
In the composition aware search tool, you can just hit the “copy space” button, and you’ll be able to find photos with spaces in the exact place you wish to put text.
Above: Copy space
The new search tool comes a little more than a year after Shutterstock first revealed a new reverse image search feature that lets you search for “visually similar” photos using computer vision techniques. Computer vision basically enables machines to search for images by processing and analyzing them on a per-pixel basis, rather than through meta data, such as keywords and descriptions.
The reverse image search feature was the first output from Shutterstock’s computer vision team, which was assembled back in 2015. While that feature in itself offered a potentially useful mechanism for finding specific kinds of photos based on “look and feel” rather than broad topics, composition aware search really does take things to the next level. Shutterstock claims around 150 million images on its platform, and it is difficult sifting through the search debris for the best photo — this is one tool that genuinely addresses the problem of information overload.
“What’s remarkable about this breakthrough is that we only trained our model to learn what things are, but our deep network learned how to represent where things are,” explained Shutterstock CEO and founder Jon Oringer. “For marketers, searching for an image with copy space using this tool will save a significant amount of time.”
Many other companies are turning to AI-supported technologies to help their users filter through the noise. Earlier this week, for example, eBay announced a new “grouped listings” view that uses machine learning and natural language processing (NLP) to filter out duplicate or similar listings in search results. For example, if you search for “microwave oven,” you would previously have been presented with multiple listings of the same microwave oven from different sellers. Now, eBay is trying to leverage algorithms to group similar listings together so that the initial search results present genuinely different products.
Shutterstock’s problem may be different from eBay’s, but the companies are looking to solve them in similar ways.
