Comma.ai launches Comma Two, a $999 kit that imbues cars with assisted driving features

Two years ago, analysts predicted that upwards of 10 million self-driving cars would hit the road by 2020. These weren’t the only bullish indicators — in 2015 and 2016, respectively, the Guardian and Business Insider proclaimed that people would become “permanent backseat driver[s],” riding in fully autonomous cars “without needing any interaction from [drivers]” from 2020 onwards.

Fast-forward to today. Despite assurances that as many as 8 million driverless cars will be added to the road in 2025, even operations like that of Alphabet’s Waymo have yet to expand beyond selected metros. The reasons are regulatory as well as technological in nature, but expense is a significant factor. Conservative estimates peg the outfitting cost at between $100,000 and 250,000 per car.

So why not go the do-it-yourself route? That’s the question posed five years ago by George Hotz, the U.S. hacker best known for developing exploits that targeted Apple’s iOS operating system and for reverse-engineering Sony’s PlayStation 3. In September 2015, Hotz founded Comma.ai with the goal of developing a semi-automated system — OpenPilot — that would improve cars’ visual perception and electromechanical motor control. (Last year Hotz abdicated the role of CEO to Riccardo Biasini, Comma.ai’s then-vice president of quality, who had joined the company after five years with Tesla.) Unlike most full-stack solutions in testing, Open Pilot is intended to replace OEM advanced driver-assistance systems, effectively imbuing cars with self-driving capabilities.

For the first time in its history, Comma.ai has an official presence at the Consumer Electronics Show. The company this week invited members of the press (including VentureBeat) to test-drive the latest version of its system — and to check out the Comma Two.

OpenPilot

OpenPilot had a bit of a rocky start. The first version’s reveal in a Bloomberg article and video prompted a cease-and-desist letter from the California Department of Motor Vehicles, which accused Comma.ai of testing a self-driving car in the state without a license. Subsequently, OpenPilot was packaged into a shippable device dubbed the Comma One, which again ran afoul of the authorities because of noncompliance with U.S. Federal Motor Vehicle Safety Standards. Under pressure from the National Highway Transportation and Safety Administration, Comma One was canceled, and Comma.ai open-sourced OpenPilot on GitHub.

Much has changed. In the roughly three years since Comma.ai made OpenPilot available to the public, over 1,500 monthly active users have racked up more than 14 million autonomous miles collectively, half of which were driven autonomously. (That’s up from 10 million miles as of May 2019.) Moreover, they’ve forked the GitHub repository over 2,300 times, adding features like automatic lane change and support for older Tesla, Chrysler, and Jeep models.

What hasn’t changed is that OpenPilot falls under the category of level 2 self-driving as defined by the Society of Automotive Engineers, meaning the human at the wheel must be driving and constantly supervising the automated features. That’s as opposed to level 4 systems from Waymo, Baidu, Uber, and others that allow drivers to turn their attention away and safely leave the driver’s seat.

Comma Two

Comma.ai previously sold the Eon Devkit, a self-contained and windshield-mounted modified OnePlus phone. It started at $599, and once OpenPilot was installed to it, automatically recognized car models by analyzing CAN network traffic. (The presence of certain CAN messages and their lengths is an indication of the car’s model, year, brand, and trim.) Companion devices dubbed the Panda and Giraffe plugged into cars’ OBD-II ports to track RPMs, MPG, cornering G-force, battery life, and more, and to provide access to the communication buses and enable the Eon to interface with the car.

The Comma Two replaces the Eon, but it retains all of the Eon’s features and then some. That said, it ditches the Panda for a single-cord solution, and it’s slightly pricier at $1,000 (though it’s available on a payment plan through Affirm).

The first batch of 300 units is available for purchase today. The units will begin shipping later in the month, Hotz says.

The Comma Two is powered via OBD-C, rather than battery, as Hotz says that was a top complaint from current Eon owners. And it has a larger mount, as well as a custom fan-based hardware cooling solution. Like the Eon, which used a camera to recognize drivers’ faces and decelerate if it detected that they were distracted, the Comma Two performs facial recognition. In point of fact, it leverages two infrared sensors as opposed to an RGB sensor, enabling it to work during nighttime.

Other highlights include front and back cameras, as well as a fourth CAN bus to connect the OBD-II port and a low-power mode that automatically shuts off the Comma Two after three days to save car battery.

The Eon was equipped with cellular service and a SIM card supplied by Comma.ai. Data was unlimited (though capped at 512Kbps), and users could elect to spring for Comma Prime, a $24 per month service that enables remote access (via T-Mobile) from anywhere and online storage of 14 days’ worth of drive data (compared with the standard three days). The Comma Two also taps cellular for connectivity and plays nicely with new and existing Comma Prime subscriptions, and it comes with a year’s worth of storage.

By default, OpenPilot uploads driving data in real time from the road-facing camera, CAN, GPS, inertial measurement unit, magnetometer, thermal sensors, and operating system to Comma.ai’s servers for machine learning training and development purposes. Much of this data can be accessed locally through the Comma Connect app for iOS and Android or through Explorer, an online dashboard of recent drives from which disengagements (i.e., instances when a driver took control from OpenPilot) can be annotated to help improve the system.

From within the app, Comma Two owners can remotely wake up the Comma Two and snap a picture from either the front- or rear-facing cameras. Additionally, they’re able to switch between multiple units on the fly and to view their real-time geographic locations.

Compatibility

OpenPilot isn’t compatible with every car under the sun. It only works with the following models and model years:

• Acura ILX (2016-2018) and RDX (2016-2018)
• Buick Regal (2018)
• Chevrolet Malibu (2017)
• Volt (2017-2018)
• Cadillac ATS (2018) and Pacifica (2017-2018)
• GMC Acadia Denali (2018)
• Holden Astra (2018)
• Honda Accord (2016-2019), Civic (2017-2019), Civic Hatchback (2017-2019), CR-V (2015-2019), CR-V Hybrid (2017-2019), Fit (2018), Odyssey (2018-2019), Passport (2019), Pilot (2016-2019), and Ridgeline (2017-2019)
• Hyundai Elantra (2017-2019), Genesis (2018), and Santa Fe (2019)
• Jeep Grand Cherokee (2016-2019)
• Kia Optima (2019), Sorento (2018), and Stinger (2018)
• Lexus RX Hybrid (2016-2019) and ES Hybrid (2019)
• Subaru Crosstek (2018) and Impreza (2019)
• Toyota Avalon (2016-2018), Camry (2018-2019), C-HR (2017-2019), Corolla (2017-2020), Corolla Hatchback (2019), Highlander (2017-2018), Highlander Hybrid (2018), Prius (2017-2019), Prius Prime (2017-2020), Rav4 (2016-2019), Rav4 Hybrid (2017-2018), and Sienna (2018)

Equally critically, OpenPilot isn’t capable of fully autonomous driving on all road types in all conditions. It can currently handle lane centering on roads without clear marking, thanks to machine learning algorithms trained with user data, and it can maintain a safe follow distance and drive in stop-and-go traffic without supervision.

But there are caveats. On all supported cars, OpenPilot’s automated lane centering and lane keep assist replace the stock systems. Only on specific supported cars does the system’s adaptive cruise control and forward collision warning slot in by default, however, and on no model does OpenPilot take over things like auto high-beam, blind spot warning, and side collision warning.

Driving experience

So how does the driving experience compare with the Comma Two? “Smooth sailing” sums it up. OpenPilot indeed maintains its lane position even when the markings aren’t obvious — or aren’t visible. In a construction site with cones on either side and on an unpaved road without dividing lines, we watched OpenPilot deftly keep centered and maintain a comfortable following distance from the car in front of it.

It doesn’t much matter whether the road ahead is straight or severely curved. OpenPilot used to scrape OpenStreetMap for road curvature and traffic data, but it doesn’t any longer — now, it’s smart enough to calculate the trajectory in real time and ensure it doesn’t speed along that trajectory too violently.

In this respect, OpenPilot is akin to Nissan’s ProPilot Assist, Volvo’s Pilot Assist, and GM’s Super Cruise, the latter of which is one of the few systems that offers hands-free driving, thanks to an eye-monitoring infrared camera. But in contrast to Super Cruise, which only engages on about 130,000 miles of U.S. and Canadian highway that GM has mapped, OpenPilot theoretically works anywhere with a discernible road.

OpenPilot can change lanes automatically, but it requires signaling from drivers before it does so — they have to first switch on a turn signal and nudge the steering wheel in the direction they’d like to go. It smoothly merged while keeping pace with the traffic ahead in our brief experience. But Hotz noted that OpenPilot won’t prevent lane changes when cars are to the immediate left or right, in the Comma Two’s blind spot.

Of course, Comma Two and OpenPilot can’t anticipate every situation. An SUV rear-ended a hatchback during our driving demo, forcing Hotz — who was behind the wheel — to tap the brake and disengage the system. And OpenPilot doesn’t handle things like red lights or stop signs — at least not yet.

Safety

Whether OpenPilot can be considered truly safe is a semantics question, in part. The driverless car industry lacks an agreed-upon metric for safety — Noah Zych, head of system safety at Uber’s Advanced Technologies Group, told Wired in an interview that miles traveled isn’t a particularly insightful measure without context like location. Derek Kan, U.S. secretary for policy at the U.S. Department of Transportation, echoed that sentiment in remarks at a conference two years ago.

Companies like Intel’s Mobileye and Nvidia have proposed mathematical models that aim to codify good habits, like giving other cars the right of way. Essentially, they’re decision-making policies in a motion-planning stack that monitor unsafe actions by analyzing real-time sensor data.

Comma.ai has a safety model of its own, which Hotz said involves several forms of regression testing. Commits aren’t merged to the OpenPilot codebase before they’ve met the test suite’s muster.

This said, Comma.ai problematically hasn’t committed to regularly publishing any sort of statistics beyond miles driven, like disengagements or accidents. California’s Department of Motor Vehicles mandates that all companies testing autonomous cars report disengagements, but because Comma.ai doesn’t have a permit to test cars, it’s exempt from this requirement.

Hotz asserts that the infrared sensors in Comma Two will improve safety substantially, as will the onboarding video customers are required to watch before they launch OpenPilot for the first time. On the subject of the former, a warning appears on the Comma Two’s screen if a driver diverts their eyes from the road for four seconds, and after six seconds OpenPilot disengages and begins to slow down. (Unclicking the driver-side seat belt or opening the car door also disengages the system.)

For what it’s worth, we encountered only one disengagement — Hotz’s stop to avoid the accident — over roughly 7 miles and 25 minutes of driving  “We pushed on nuance [with this latest version of OpenPilot],” said Hotz. “Intersections have no lane lines — it’s unclear where you should go. [OpenPilot] follows the path that the human would follow.”

Backend improvements

On the backend side of the OpenPilot equation, there’s a new simulation environment called the Small Offset Simulator that loads in real-world routes and simulates deviations. “You can’t just take a lot of data and train a model to mimic the data, because that model doesn’t realize that if it takes action, the world will change,” said Hotz. “The only way to really do that is to build a simulator.”

Using a technique called projected geometry and generative adversarial networks (GANs), or two-part AI models comprising generators that synthesize data samples and discriminators that attempt to distinguish between the synthesized samples and real data, the simulator redraws scenes meters to the left or right or with curvature and other perturbations. Hotz said the models within the Small Offset Simulator are now training on 200,000 minutes of driving data collected by Eon users.

It’s similar — but not quite the same — as the simulation environments deployed internally by Uber, Waymo, and others. GM’s Cruise, for instance, leverages a replay approach that involves extracting real-world sensor data, playing it back against the car’s software, and comparing the performance with human-labeled ground truth data. Cruise also engages in planning simulation, which lets the company’s data scientists create up to hundreds of thousands of variations of a scenario by tweaking elements like the speed of oncoming cars and the space between them.

Future developments

Going forward, Comma.ai plans to release new hardware on a roughly yearly cadence — Hotz believes its business model of selling devices at profit and eating the cost of software development is sustainable. (Comma.ai has raised $8.1 million in venture capital to date across two funding rounds, the most recent of which closed in April 2018.) As for OpenPilot, achieving a better end-to-end driving experience remains an acute area of focus for the engineering team.

“What we want to do is move away from [lead car following] and move to an end-to-end longitudinal [model],” he said. “One of the problems we as a company have been trying to convey is when it works well, you don’t even notice it. Our company mission is to solve self-driving cars while shipping intermediaries.”

But Comma.ai isn’t discouraging rivals from cloning its hardware and piggybacking off of OpenPilot. In fact, Hotz encourages it, just as he encourages those with proprietary solutions to compete with Comma.ai.

One such challenger — former Yahoo CTO John Hayes’ Ghost Locomotion — emerged from stealth last November promising an aftermarket self-driving kit to retrofit existing cars. Another — Cambridge, U.K.-based Wayve — claims its driverless cars self-improve by learning from safety driver interventions. Mobileye doesn’t sell a consumer product, but like Comma.ai its driverless tech leans on cameras for autonomous decision-making. And then there’s Tesla, which recently released a preview of an active guidance system (Navigate on Autopilot) that navigates a car from a highway on-ramp to off-ramp, including interchanges and lane changes.

Hotz is unconcerned about the challenges ahead: “Mobileye is the Windows [of autonomous driving] that’s going to do Microsoft-style business development deals with big companies, Tesla’s the Mac that’s going to ship sexy hardware to consumers, and we’re the Linux,” he said. “Everybody should use us for whatever they want.”