Watch all the Transform 2020 sessions on-demand here.
Nvidia Research today unveiled GauGAN, a generative adversarial AI system that lets you create lifelike landscape images that never existed.
The research and a demo of the new system were shown off today at the GPU Tech Conference (GTC) in San Jose, California.
GauGAN builds upon learnings from the Pix2Pix system introduced last year that can render virtual worlds, said Nvidia VP of applied deep learning research Bryan Catanzaro, but Pix2Pix can’t paint landscapes because doing so leaves artifacts in the resulting image.
GauGAN’s neural network is trained with one million open source Flickr images and imbued with an understanding of the relationship between more than 180 objects like snow, trees, water, flowers, bushes, hills, or mountains.
June 5th: The AI Audit in NYC
Join us next week in NYC to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.
That understanding of how objects relate to each other means a tree next to water will show a reflection, or when the season changes and there’s snow on the ground, trees will be depicted without leaves.
Style transfer is also possible so an image can adopt a warm sunset glow or display the cooler lights of a city skyline.
The GauGAN app uses a segmentation map, which acts sort of like a coloring book that describes where objects are but provides no detail, as seen below.
A paper by Nvidia principal research scientist Ming-Yu Liu and others that details the creation of GauGAN and its spatially adaptive denormalization method for photo manipulation was released today. The paper was also accepted for oral presentation at the Computer Vision and Pattern Recognition (CVPR) 2019 conference to be held in June in Long Beach, California.
Also making its debut today is the Nvidia AI Playground, a website where people can tinker with a variety of trained neural nets like GauGAN that use powerful AI to distort visuals or create lifelike images.
GauGAN is the latest reality-bending AI system from Nvidia, creator of deepfake tech like StyleGAN, which can generate lifelike images of people who never existed, and Nvidia open sourced last month.
Responding to a question of Nvidia releasing another system that can make people question what’s real, Catanzaro said that as a researcher, “this has been my dream” because the tech necessary to achieve such results represents progress — but on the other hand, it is tech that some people may misuse.
“Personally, I believe it’s a trust issue and not a tech issue,” Catanzaro said.
Catanzaro said he’s most excited about GauGAN being adopted by video game designers creating the landscape of virtual worlds and envisions applications of the technology for creatives who use storyboards to lay out their creations.
