Watch all the Transform 2020 sessions on-demand here.
Nvidia has developed a method to train robots to carry out actions by first observing human activity. In initial applications, robots learned to pick up and move colored boxes and a toy car in a lab environment, using a Baxter robot.
Learnings from such research will be used to retrain robots and create robots that can work safely alongside people in industrial settings and homes.
“In the manufacturing environment, robots are really good at repeatedly executing the same trajectory over and over again, but they don’t adapt to changes in the environment, and they don’t learn their tasks,” Nvidia principal research scientist Stan Birchfield told VentureBeat in an interview. “So to repurpose a robot to execute a new task, you have to bring in an expert to reprogram the robot at a fairly low level, and it’s an expensive operation. What we’re interested in doing is making it easier for a non-expert user to teach a robot a new task by simply showing it what to do.”
The system has a series of deep neural networks that perform perception, planning, and control, and these networks are trained entirely on synthetic data.
June 5th: The AI Audit in NYC
Join us next week in NYC to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.
“There’s sort of a paradigm shift happening in the robotics community now,” Birchfield said. “We’re at the point now where we can use GPUs to generate essentially a limitless amount of pre-labeled data — essentially for free — to develop and test algorithms. And this is potentially going to allow us to develop these robotics systems that need to learn how to interact with the world around them in ways that scale better and are safer.”
The findings were shared today at the International Conference on Robotics and Automation (ICRA) taking place this week in Brisbane, Australia.
The new AI system was made with help from the Nvidia robotics research lab. First announced late last year, the lab now has six employees and is preparing to open up offices adjacent to the University of Washington in Seattle this summer.
The research lab will continue to work with the robotics community and the in-house team at Nvidia to explore the use of synthetic datasets for training AI systems, Nvidia’s head of robotics research, Dieter Fox, told VentureBeat.
Such knowledge could be used to strengthen the Isaac SDK, a framework for training robots with simulations first introduced in May 2017.
“Nvidia actually has been working in that domain for quite a while in the gaming context, for instance, where it’s all about setting up 3D virtual environments that are photo-realistic and give you some kind of content modeling. And what we want to do is also work with all these teams that have all this expertise, but help them expand it in a way so that it becomes better applicable in a robotics setting,” Fox said.
Research like the kind released today will be central to the creation of the next generation of robots, Fox said.
“We’re talking about robots that have to open doors, open drawers, pick up objects, move them around, even physically interacting with people, helping them — for example elderly people in the home,” he said. “These robots need to be able to recognize people, they need to see what a person wants to do, they need to learn from people, learn from demonstration, for example, and they also need to be able to anticipate what a person wants to do in order to help them.”
Nvidia joins a growing number of companies, like Google and SRI International, that are interested in the development of AI systems with a kind of environmental awareness, or, as Google AI chief Jeff Dean put it, more “common sense.”
To read more about the new robotics research, see this Nvidia blog post.
