Watch all the Transform 2020 sessions on-demand here.
Has artificial intelligence ever been put to a more silly human endeavor? Gfycat said it is launching Gfycat AI to apply machine learning to improve animated GIFs (the silly moving images associated with Internet memes in the graphics interchange format, or GIF) with features like better face recognition and video game character identification.
Gfycat AI has three distinct machine learning projects: Maru, Felix, and Angora. They’re all named after cats, which are an Internet obsession.
Project Maru uses facial recognition technology to identify people in a GIF. So far Project Maru has identified and tagged over 3,000 celebrity faces in Gfycat’s GIF database, with over 1.2 million GIFs tagged. Gfycat uses an open source model with an accuracy of 99.38 percent on the standard LFW (“labeled faces in the wild”) face dataset.
Above: Project Angora GIF
Above: Project Angora
Since gameplay GIFs are popular on Gfycat, Maru is also able to detect photorealistic video game characters, such as those from Overwatch. Maru can also identify when a GIF shows footage from a particular video game; for example, it’s able to accurately identify and tag GIFs of video games such as The Last of Us.
June 5th: The AI Audit in NYC
Join us next week in NYC to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.
The tech behind Maru has applications beyond improved tagging and search results; for instance, analysis of facial recognition results can be used to determine which celebrities and public figures are trending.
Gfycat performed an analysis of the most popular Overwatch characters on the platform and was able to gain insight into Overwatch fans’ viewing habits. Specifically, results from Gfycat AI show that Overwatch fans’ favorite characters to play are different from their favorite characters to watch. The top five played characters are a mix of male and female, while the top GIFfed characters are predominantly female.
D.Va, the No. 1 top GIFfed character, doesn’t even make it to the top five most played characters. Clearly users have different priorities when watching gameplay than they do when playing the game.
Above: Project Angora
Project Felix extracts captions from a GIF to improve tagging and search. Many GIFs uploaded to Gfycat were originally created in a different software application, and in those cases, captions aren’t entered into the Gfycat database. Using the data gleaned from Felix, Gfycat is able to better understand the emotions and memes that are being shared at a given time. In the future, Gfycat will be able to also better understand score data and achievements from gaming-related GIFs.
“We’re excited about the possibilities opened by these innovative AI projects,” said Gfycat CEO Richard Rabbat, in a statement. “With the data we’re gathering from Gfycat AI, we’re able to gain more insights into user behavior, improve search results, and improve the quality of GIFs on the Internet.”
Project Angora uses machine learning to automatically search the web for low-quality GIFs and recreate them in HD with higher frame rates. Gfycat is capable of serving GIFs in up to 8K quality. But most GIFs are low-quality and only support 256 colors. When users find or create GIFs from other sources and upload them to Gfycat, often those GIFs are much lower quality than the platform is capable of.
Project Angora uses computer vision algorithms to locate the high-quality video source for low-quality GIFs. Once discovered it produces a significantly higher quality version with Gfycat’s HD GIF support. Project Angora uploads about 15,000 GIFs a day on Gfycat and has to date created 2 million high-quality GIFs available on the platform.
“As investors in two popular gaming companies, Twitch and Niantic, we at Alsop Louie are excited that Gfycat’s AI initiatives are shaping our understanding of how gaming fans interact with their favorite games,” said Ernestine Fu, partner at Alsop Louie Partners and Gfycat board director. “The results we’ve seen so far are just the beginning.”
