Nvidia is launching its EGX Platform to bring real-time artificial intelligence to the edge of the network. This means AI computing will happen where sensors collect data before it is sent to cloud-connected datacenters.
“There’s a massive change in the computing industry being driven by growth of [internet of things] sensors,” said Justin Boitano, senior director of enterprise and edge computing, in a press briefing. “There are cameras for seeing the world, microphones for hearing the world, and devices being deployed so machines can detect what is happening in the real world.”
But this also means there’s an exponential increase in the amount of raw data that has to be analyzed.
“We will soon hit a crossover point where there is more computing power at the edge than in datacenters,” Boitano said.
June 5th: The AI Audit in NYC
Join us next week in NYC to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.
Nvidia and its customers will offer a new class of servers for instant AI on real-time streaming data in markets such as telecommunications, medicine, manufacturing, retail, and transportation. Nvidia showed the platform at the Computex event in Taiwan.
Nvidia EGX will provide accelerated computing at the edge for low-latency transactions, or those with minimal time delays between interactions. This will enable real-time reactions to data pouring in from sensors for 5G base stations, warehouses, retail stores, factories, and beyond.
“AI is one of the most important computing challenges of our time, but CPUs have not been able to keep up,” Boitano said.
Nvidia created EGX to meet the emerging needs of AI applications serving what will eventually be trillions of devices streaming continuous raw sensor data.
The platform is designed for emerging, high-throughput AI applications on the edge, where data is sourced to achieve instantaneous or guaranteed response time while reducing bandwidth to the cloud. By 2025, 150 billion machine sensors and IoT devices will stream continuous data that will need to be processed. That will be orders-of-magnitude more data than is produced by people.
Edge servers will be distributed throughout the world to process data from these sensors in real time.
Nvidia EGX is a “hyperscale cloud-in-a-box” — combining the full range of Nvidia AI computing technologies with Mellanox security, networking, and storage technologies. Companies in the largest industries — manufacturing, retail, health care, and transportation — can use EGX to deploy AI quickly and securely, from edge to cloud.
Nvidia said EGX is scalable. It starts with the tiny Nvidia Jetson Nano — which in a few watts can provide half a trillion operations per second (TOPS) of processing for tasks such as image recognition — and spans all the way to a full rack of Nvidia T4 servers, delivering more than 10,000 TOPS for real-time speech recognition and other real-time AI tasks.
Nvidia EGX is built for enterprise and industries and is optimized for running enterprise-grade Kubernetes container platforms such as Red Hat Openshift. These container platforms have been tested with Nvidia Cloud Stack, an optimized software suite, to simplify setup, provisioning, and management of GPU-accelerated infrastructure for running Tensor RT, Tensor RT Inference Server, and NGC Nvidia GPU Cloud registry.
By combining Nvidia EGX, Red Hat OpenShift, and Nvidia Edge Stack, enterprises can easily stand up a state-of-the-art edge to cloud infrastructure, Nvidia said.
To enable hybrid cloud computing, Nvidia EGX-powered systems and devices can connect to cloud IoT services. Customers can remotely manage their service from AWS IoT Greengrass and Microsoft Azure IoT Edge.
“Azure IoT Edge helps customers deploy cloud service to their IoT devices quickly and securely,” said Sam George, director of Azure IoT Edge, in a statement. “We look forward to supporting Nvidia’s EGX edge platform on Azure IoT Edge devices so that customers can deploy AI workloads targeting EGX-compatible hardware.”
Early adopters include more than 40 industry-leading companies and organizations, from BMW to Foxconn.
