Watch all the Transform 2020 sessions on-demand here.
Facebook AI Research (FAIR) today unveiled PyTorch3D, a library that enables researchers and developers to combine deep learning and 3D objects.
As part of the release, Facebook is also open-sourcing Mesh R-CNN, a model introduced last year capable of rendering 3D objects from 2D shapes in images of interior spaces. PyTorch3D was inspired by Mesh R-CNN and recent 3D work by Facebook AI Research, FAIR engineer Nikhila Ravi said.
Working in 3D is important for rendering 3D objects or scenes that appear in mixed reality or virtual reality. It can also be used to tackle AI challenges like robotic grasping or helping autonomous vehicles understand the position of nearby objects.
PyTorch3D comes with frequently used 3D operators and loss functions for 3D data and a differentiable mesh renderer for creating 3D objects. PyTorch3D also has a differentiable rendering API, some CUDA support, and heterogeneous batching capabilities unavailable in any existing 3D library, Ravi told VentureBeat in a phone interview.
June 5th: The AI Audit in NYC
Join us next week in NYC to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.
“With PyTorch3D, researchers can input all these functions and use them with the existing deep learning system in PyTorch and it greatly reduces on the time to work on 3D planning research, which requires a lot of expertise in order to get started, and we want to try and reduce that ramp-up time,” she said.
PyTorch3D uses meshes, a data format for interoperability of vertices and faces that make up 3D objects, and can use a patch tensor to collapse all vertices for meshes in a batch into a single tensor involved with batching, a common process for deep learning research.
The premiere of PyTorch3D follows the launch of the PyRobot robotics framework last year, and FAIR 3D research that extracts characters from real-world videos.
