Nvidia’s Jensen Huang goes off script with CES 2018 Q&A

VentureBeat: Right.

BFGD is G-Sync, 120Hz, and so there’s literally no stutter. If it’s running at 120Hz there’s no stutter, of course. You can’t feel it. If there’s any changes in framerate, G-Sync kicks in and there’s no stutter. It’s just smooth, low latency, high dynamic range, huge format. I can’t imagine a better way to game. The response from the reviewers and gamers I’ve seen is completely over the top. People just love it. You’re engulfed in this virtual world. Life is good.

What we did was we also thought—every display and every monitor should be a smart display, a smart monitor. How can you have something that’s electronic that isn’t connected to the internet? That somehow isn’t a streaming device? If it’s a display, it should be a streaming device. We took the Shield computer and we designed it right into the BFGD. You take it out of the box and it’s a wonderful streamer, the best on the planet. It supports Android TV. If you’d like to download games to play, boom, it comes from the Android store. If you’d like to stream games from GeForce Now, our cloud gaming service with the supercomputers up in the cloud that processes all the games, streams it to you, all you have to do is subscribe and enjoy games. Now you can enjoy games from the Android store, from GeForce Now, and from your PC.

We announced all that. We also added another platform to GeForce Now, our supercomputer at Nvidia with a whole bunch of GPUs in it. It’s running games and streaming them to devices that don’t have enough capability to play games natively. The performance is fantastic. It has our latest generation of GPUs. We’ve been supporting the Mac. The response has been incredible. We have a long waiting list. Recently we also opened it up for PC. Long term, we’d like to be able to make it possible for you to have incredible gaming experiences no matter what device you’re on. It could be a phone, a Chrome device, anything. We just want you to be able to enjoy great games, so long as you have a display.

The GeForce side did amazing work. Then we did one more thing, GeForce Experience. We updated GeForce Experience, our software platform that helps you enjoy games even more. The first thing it does is it automatically sets up your computer for all the various configurations and settings, so the moment you launch your game, you’re already enjoying it. But we also make it possible for you to share, so you can capture that amazing moment. It’s running in the background. It’s storing video, recording your gameplay, without affecting your performance. When you have an amazing moment, that’s been captured to share with all your friends.

We also made it possible to turn video games into the world’s first virtual camera. You can go in your video game and take amazing pictures of the scenery and the vistas, maybe the moment, and apply different effects to it. We call that Ansel. We have Shadow Play, which is recording you all the time, we have Ansel, and then we have this new thing we call Freestyle. Now that you’re playing your game, you can even apply special effects to it. Our filters, our image processing, is so powerful now that we can make it possible for you to enhance the imagery, or even change the mood of the imagery. Maybe for Battlefield, you could move it to a look that’s more vintage, more WWII. There are lots of ways we can allow you to enhance imagery. The response to that is incredible. It won Gadget of the Show at CES. It’s not even a gadget. It’s software. But it’s a great gadget as well. People love it.

That’s gaming. On the other side, we talked about the largest consumer electronics device, cars. We announced several things. We announced that Xavier, the most complex SOC the world has ever seen—2000 Nvidia engineers worked for three years already, and there’s another year to go, working on this incredible chip. This is the largest SOC the world’s ever made, 9 billion transistors. Think about all the processors inside. It has an eight-core custom CPU. We call it Carmela. ARM 64. Level one, level two, level three cache, all ECC, all parity. Designed for resilience. Server-class CPU.

It has the Volta GPU in it, with the brand new tensor core instructions. It has a brand new deep learning accelerator to complement our GPU, called DLA. 10 teraflops of performance. It has a brand new computer vision processor called PVA, so we can process stereo disparity, the parallax of your displays, to extract information like depth. Stereo disparity and optical flow, how pixels are moving from scene to scene. It has a brand new ISP, an image sensor processor, that allows us to process 1.5 gigapixels per second, so all of the cameras and sensors around your car can be in full high dynamic range. It has a brand new video processor that does encoding and decoding at a level where every single camera around your car can be processed and recorded, like a black box, while performing optical flow processing to it.

Did I get it all? It’s a crazy number of new processors. The entire processing pipeline of a self-driving car, from image processing, sensor calibration, perception, localization, and pathfinding—every single line of code of the entire self-driving car stack can run on Xavier. A one-chip self-driving car. We took all of the processing that was needed from our Drive PX2, a four-chip Pascal solution, and we compressed it into one. 9 billion transistors, 12nm finFET. Silicon is back. It’s working great. I showed it driving a car on stage.

We have 320 developers, from tier one OEMs to startups to taxi service companies to mapping companies to research organizations, that are dying to get their hands on Xavier. We’ll start sampling it this month. That’s our first announcement.

We also announced–the single most important feature of a self-driving car is safety. Safety is not just about trying really hard and being really careful. You have to design technology that makes it possible for a computer to be safe. If you haven’t had a chance to think about safety, this is a fun area. Over the last two or three years, I’ve dedicated myself to this field. I’ve learned a lot. We changed the architecture and infused safety technology into our drive stack, from the architecture through the chip design, through all of the software, through all of the algorithms, into the system, into the cloud.

You have to solve three fundamental problems. In order to do functional safety, number one, you have to have safety of the intended features. It’s called SotIF. Whatever you decide is the intended outcome, you have to somehow validate that you’re designing something according to that. Number two, it has to be resilient to hard failures, hard faults. If a wire were to break, if a chip wire open circuits, if a solder join wears out, a hard failure, it has to be resilient to that. And it has to be resilient to soft failures. It could be a noise glitch. The temperature’s too high. The memory forgot a bit. We got hit by an alpha particle somewhere. All of a sudden we have soft failure.

These three fundamental failure types – functional failure type, otherwise known as systematic failure, hard failure, and soft failure – the architecture in totality has to be able to deal with them. In the final analysis, what you’re looking for is the ability to have redundancy and diversity. Everything you do has to have backups. Everything you back up with has to be different from what you backed up originally. Another way of saying it is, I need to be able to do computer vision in multiple ways. I need to do localization in multiple ways. I need to sense my environment in multiple ways. Not just many ways, not just more than one way. It has to be redundant and diverse.

We’re going to design future cars the way people design airplanes. Except we have to use so much technology and ingenuity to reduce its cost and its form factor. We can’t afford to have a jet plane. That would be great if could. It would be the brute force way. But unfortunately the cost is too high for society to bear. We have to apply all kinds of technology and innovation to do that. We announced that the Nvidia drive stack will be the world’s first top to bottom functional safe drive stack, ISO 26262, with full functional safety. It’s a big achievement.