Hands Off the Wheel in San Diego: A Comma Two Demo Drive and a Tour of comma.ai's Hardware Shop

Lex Fridman introduces the video as a demo drive and office tour with George Hotz in San Diego after their podcast chat. Hotz drives toward the office looking for a stretch where the Comma Two can be enabled, noting that the area is not where it shines. He says the product does not yet handle red lights and stop signs, and he does not want to ship that until it is a holistic end-to-end system.

Engagement works through the car's own cruise control controls. Hotz calls a long stretch a good test of whether a level-two system is any good, claiming that Super Cruise cannot do it because it is not good enough with road roll, while Autopilot and comma can. San Diego's roads and lane markings are, he says, pretty mediocre, like the rest of California.

Asked where he finds the most joy in using it, Hotz points to long trips. A two-hour drive to Los Angeles can be done almost without a disengagement, and you can sit and watch, which he calls a great time.

Hotz says the system does not need perfect lane markings. Because it is end to end, it can even make lane changes on roads that have no markings. Before a three-lane move near a police officer he takes control himself, then re-engages on a curvy, banked road that he says only comma and Autopilot can do, because a model has to detect the banking.

The longitudinal side, acceleration and braking, is currently left to the car. Hyundai, Hotz says, has quite a competent adaptive cruise control, probably the best, so comma has not felt the urge to take over the gas and brakes. He notes a few cases where comma would do better and more holistically, and a moment where the stock system brakes too harshly compared to a human.

Fridman asks how hard the smoothness is to achieve, and Hotz answers "hard," describing five years of engineering. Lane lines are still detected but used more as a cue. The plan is to move all longitudinal control to openpilot once there is end-to-end longitudinal, in the same way the lateral control is already end to end.

Hotz explains why comma put far more effort into end-to-end lateral control. Car makers have built competent longitudinal control, and comma has a competent one using old-school policies, but steering has much more subtlety. For example, when a lane splits, should the car follow the split, stay in the center or stay to one side? Making all of it smooth and not alarming is the goal.

On safety, he says he feels confident keeping his hands off the wheel because comma's torque limit is much lower than Tesla's, so the system cannot do anything crazy. The team runs injection tests where one person sits in the seat with a joystick while the other keeps hands off the wheel and doesn't look. The joystick person can jam it in any direction, and the other must be safe in that maneuver or take over.

Fridman finds the stretch impressive, and Hotz says it is not even the high end. On city streets, he says, the lack of end-to-end longitudinal control makes it not really useful as a product. It is cool that it can do it, and comma does not gate it off, but most of the value is on highways and on roads with one lane each way and a double yellow line. He says he used it in the Colorado mountains so he could look at the scenery.

On an intersection with no lines, Hotz notes the car's placement is almost human-ideal. After about a minute of city driving that was perfectly human with nothing sketchy, he explains that disengaging on gas makes handoffs easy, because the sound makes it clear when control changes. Fridman likes the sounds and visuals that show whether the system is on, and says the car communicates uncertainty with a small wobble. Hotz adds that humans wobble more than people think.

Back at the office, Hotz shows the dashboards. The company is almost at 30 million openpilot miles, and charts show daily, weekly and monthly users and how many devices are on the network in the last hour, with a world map where the US leads and Korea, Canada, Taiwan and Japan follow. About 50% of miles are engaged and about 30% of time, and disengagements split roughly evenly between cancel, gas and brake. Most data now comes from Comma Twos, which Hotz calls the big breakout and which is profitable.

A whiteboard states the mission: solve self-driving cars while delivering shippable intermediaries. A list of tasks includes moving everything to PyTorch, and a vision board for openpilot 1.0 that Hotz says will come to the Comma Two through over-the-air updates. He jokes that the compute cluster is not a data center for legal reasons.

The tour continues to the hardware area, where Comma Twos spend about 24 hours in a stress test checking heat, light sensors, cosmetics and screens. The devices start as retrofit phones that get boards, soldered parts, a heat sink and fan, and then cases, and a batch of about 100 is being tested. Fridman also meets a machine learning teammate, Chris, known as a famous openpilot YouTuber.

In the loud, hot room for 3D printers, Hotz says they just got about 20 printers in the last few months from places like Poland, Germany and Peru. Each printer has a name, and Fridman gets one named after him. The room also hosts the compute cluster, where the team jokes about the "sound of productivity."

The cluster is covered to keep out bugs and dust. Hotz describes three racks of CPU machines with two petabytes of spinning disks, and GPU machines with V100s, noting that 3080s are only in desktop computers for non-business use. Finished harnesses are assembled by hand and stacked ready to ship, which Fridman calls legit.

Fridman mentions that Hotz took him on a ride and he was surprised how well it worked, including on a highway and a bit in the city. The tour ends with Hotz saying the team are big Elon fans and that Elon's success is theirs, comparing it to iOS and Android succeeding together.