Skip to main content

Phantom Auto uses humans to remotely control self-driving cars in a pickle

https://www.youtube.com/watch?time_continue=14&v=ZHzsMbBjI9w

Self-driving cars might not respond perfectly in every driving situation, which could be deadly, as passengers aren’t likely to be paying attention or able to take over in time. So Phantom Auto has come up with a solution. When the situation gets really tough for a self-driving car, Phantom Auto switches the control to a remote human driver, who can use video game equipment to safely control the vehicle.

Shai Magzimof, CEO and cofounder of the Mountain View, California startup, said in an interview with VentureBeat that he was inspired by playing racing video games. After looking into programming artificial intelligence for self-driving cars, he concluded it would be really hard to achieve perfection. And any amount of error could be disastrous for passengers.

About 78 percent of the public say they do not currently trust self-driving cars, but 59 percent are willing to buy cars with some autonomous features.

“Our idea is to support an autonomous vehicle when the AI isn’t yet perfect and it has trouble getting around road construction or dealing with weather,” Magzimof said. “This is exactly like playing a video game. But here you are using video game equipment to drive a real car. We believe AI will ultimately solve the problem. But until then, we can support the cars. We bring the human touch back to self-driving cars.”


June 5th: The AI Audit in NYC

Join us next week in NYC to engage with top executive leaders, delving into strategies for auditing AI models to ensure fairness, optimal performance, and ethical compliance across diverse organizations. Secure your attendance for this exclusive invite-only event.


This reminds me of a fireside chat at our GamesBeat Summit 2017 event in April, where Rod Chong of Slightly Mad Studios talked with AImotive CEO Laszlo Kishonti about using simulations to test self-driving cars.

Above: Shai Magzimof shows off a Phantom Auto self-driving and remotely piloted car.

Image Credit: Dean Takahashi

Of course, if you crash in a video game, nothing happens, but in a car with real people as passengers, you can’t goof off. That’s one reason Magzimof wants to limit the speed for a remote operator to 25 miles per hour.

Magzimof’s 10-person team created a system that allows the self-driving cars to be auto-piloted most of the time. In exceptional situations, a human driver with video game equipment, such as a steering wheel and driving pedals, could take over the car. I took a ride in one of Magzimof’s test vehicles.

At first, Magzimof showed the vehicle being piloted remotely in a parking lot, where the vehicle was being controlled by software engineer Ben Shukman, who was sitting in front of a screen with a steering wheel. Shukman remotely drove the car and backed it up, and then Magzimof and I got in. He proceeded to drive us around the block at speeds under 25 miles per hour, which Magzimof said is the current limit for Phantom Auto’s human remote drivers. As we drove, we came across a construction sign that could have given a self-driving car trouble. But it was no big deal for a human.

Magzimof believes state regulators at the California Department of Motor Vehicles will favor this approach, as it provides an extra level of safety for cars. One of the reasons this is important is that human drivers in the future won’t be attentive. They may be sleeping, playing a game, or doing something else with the expectation that the self-driving car is in control, and they won’t be prepared to take over in an emergency.

Above: Phantom Auto could replicate lots of these stations for a fleet of self-driving and remotely piloted cars.

Image Credit: Phantom Auto

Phantom Auto gets around that problem in part by using low-latency communication through a smartphone. The company makes sure that a solid connection is established between the car and the remote human driver. So when the remote human driver makes a turn using the Logitech steering wheel and gas pedal, the car actually responds in real time.

With this safety solution, Magzimof believes that self-driving cars will be approved for wider testing and faster deployment.

He thinks of the company as a remote-control-as-a-service provider for autonomous vehicles. His cofounders include experts in robotic vehicle control. They have created the remote operator service, an in-vehicle communication device, and an applications programming interface for real-time assistance and guidance. For the operator, they use high-end video game equipment, such as Logitech’s steering wheel and foot pedals.

And while self-driving cars could eliminate a lot of jobs, Magzimof thinks he can create jobs for the former drivers, once the self-driving cars take over.

“I want to hire the Uber drivers that have a five-star rating,” he said.

Above: Phantom Auto has human driver Ben Shukman remotely piloting an autonomous car.

Image Credit: Phantom Auto

The human in the car could summon the driver to take over in a challenging situation, such as a snow-covered road that is hard to navigate, or the driver could take over when danger is imminent. Of course, that means the remote driver will have to be paying attention. For the remote driver, the job could be fun, since it’s basically playing a video game, driving a car around on the screen — except it’s a real car.

If the car has a malfunction, the remote driver could call for a replacement self-driving car to pick up the stranded passenger. In that sense, Phantom Auto could serve as an emergency roadside service company. And one remote driver should be able to manage multiple cars.

“A car may need support, and the remote driver connects immediately,” Magzimof said. “If a policeman stops a car, who will talk to him? You have to have a remote driver communicating with him.”

Magzimof started the company earlier this year. He had been looking for something to do after he sold his previous startup. That company was Nextpeer, a platform that enabled mobile games to add multiplayer matches. Viber acquired Nextpeer for $9 million in 2015. For Phantom Auto, Magzimof raised money from Maniv Mobility, Wolfson Group, OurCrowd, Tectonic Capital, and Joseph Grundfest.