Human-Centered or Fully Autonomous? Lex Fridman's Case for Collaboration Between Driver and Machine

Fridman starts with the utopian case. About 1.3 million people die in car crashes globally each year, with roughly 35,000 to 40,000 in the United States, and he calls designing AI systems that save lives the main focus at MIT. Autonomous vehicles could remove what NHTSA calls the four Ds of human folly: drunk, drugged, distracted and drowsy driving. Shared mobility could also eliminate car ownership, cutting the cost of getting from A to B by an order of magnitude.

The dystopian view includes job losses, especially in trucking and transportation. It also includes the philosophical and ethical problem of an algorithm, written by an engineer, killing someone who would have lived had a human been in control. Fridman adds security, since cars run millions of lines of code that a hacker could manipulate.

He says the truth lies in the middle and cautions that our intuition about what is easy or hard for AI is flawed. People are extremely good at driving, and intuition has to be grounded in the data, annotation and algorithm. He urges skepticism toward company promises of vehicles two or more years out, and says the real proof is public-road testing or consumer availability.

He cites Rodney Brooks's prediction that a driverless taxi service offering arbitrary pickup and drop-off in a major US city will arrive no earlier than 2032, and in multiple cities by 2045. Fridman says he agrees but also notes that technology adoption has been getting faster over the last century, so a brilliant idea could change everything overnight.

Fridman walks through the SAE J3016 levels, from no automation through cruise control, adaptive cruise control and lane keeping to conditional, high and full automation. He says they are useful for legal discussion, policy and media reports but not for designing systems. He admits he is personally confused about level 3.

His alternative is two systems, since every system involves a human to some degree. In human-centered autonomy the human is involved and responsible, and in full autonomy the AI is fully responsible and therefore legally liable. For human-centered systems he asks how often the system is available, and how many seconds a driver gets to take over. In today's vehicles that time is zero guaranteed, with no standard.

He insists that human-centered systems will fail at some point, and the human must catch it. For full autonomy, teleoperation doesn't count because a human is still in the loop, and the "ten second rule" doesn't remove responsibility, since the driver could be asleep or having a heart attack. A fully autonomous system must find a safe harbor, even if only a parking lot.

Fridman places levels 1 to 3 under human-centered autonomy and levels 4 and 5 largely under full autonomy, with a crossover. As an example, he describes Waymo's recent trips in Phoenix with no safety driver or engineer, and calls it an incredible accomplishment and a true full-autonomy trip. He also says that today the human generally overrides the AI, and society isn't ready for a car to say that you're drunk and can't drive.

Fridman compares the two paths using a chart in which easier means the system doesn't need 100% accuracy and harder means every miss, however small, can cost lives and huge amounts of money. For localization, scene understanding and planning, human-centered systems still do perception and control. But because a human is there to catch failures, they can be wrong rarely.

The hard part for human-centered systems is human-robot interaction: perceiving the driver and communicating with them. The system must help the driver learn where its flaws are, and bring their attention back to the road. He says rarity makes this the challenge, because a rarely flawed system must still get the interaction right.

For full autonomy, the hard piece is communication with the outside world, including pedestrians, jaywalkers and cyclists. Fridman jokes he wishes Waymo luck in Boston, where pedestrians will take advantage of a car that doesn't assert itself. A driver in a human-centered car can roll down the window and yell, but building that assertion into an AI is extremely difficult.

Fridman notes that most human-factors literature, including from aviation, is skeptical of the human-centered approach. People over-trust automation, get lazy and can't re-engage when it fails, and the better the system gets, the worse the problem. He cites Chris Urmson, founder of Google's self-driving program and now co-founder of Aurora, who said Google tried shared autonomy and gave it up when engineers fell asleep.

To test this, his team instrumented 25 vehicles, 21 of them Tesla Autopilot cars, with two HD cameras on the driver, one on the road and full vehicle data. They have collected over 300,000 miles and more than 5 billion video frames, which he says is several orders of magnitude bigger than the computer vision datasets people usually work with. They analyze it with computer vision, tracking glance, drowsiness, emotion, hands on wheel and body pose.

About 33% of the miles in the data were driven on Autopilot, which he calls a remarkable adoption rate and says is much higher than the literature would suggest is safe. Glance classification shows that where drivers look does not significantly change between Autopilot and manual driving, and drivers don't start texting or move to the back seat. The data has no crashes or near crashes in Autopilot, and Fridman says this shows promise for the human-centered approach, which is mostly highway driving.

Fridman notes that the course's guest speakers all work on the full autonomy side, the side roboticists know how to solve, while he represents the human-centered side. His belief is that solving perception and control well enough for full autonomy is extremely difficult and decades away. Meanwhile, he says, human-robot interaction is how AI systems can get on the road.

The counterintuitive step is to let AI systems reveal their flaws, as friends reveal flaws to each other, even though this may not appeal to a car company. Autopilot does this in a limited way by letting drivers turn it on almost anywhere. People try it on rural roads, in heavy rain or snow, or on curvy roads with poor markings, to learn by touch where it will fail.

Fridman concludes that autonomous vehicles can be seen as personal robots you build a relationship with, and that human-robot interaction is the key problem, not perception and control. Flaws on both sides must be perceived and communicated: computer vision detects the human, and the car's displays or voice reveal what it doesn't see. With that in place, perception and control can focus on being good without needing 99.99999% accuracy.

Radar, which works with electromagnetic waves, and ultrasonic, which works with sound, are cheap and perform well in rain and snow. Fridman calls radar the old trusted friend, available in most vehicles with any autonomy. Its drawback is low resolution, and ultrasonic is used mostly for parking assistance and blind-spot warning because its range is so short.

Lidar provides extremely accurate depth information and a high-resolution 360-degree map, and has been a source of ground truth. Its downside is cost, and it does poorly in rain, fog or snow. Cameras are cheap, high resolution and everywhere, with the longest range in clear conditions and many orders of magnitude more data available than for lidar.

Cameras also see the world as we do, with lane markings, traffic lights and signs designed for visible light. Their weaknesses are noisy depth estimation, poor performance in the dark and in extreme weather, and sensitivity to lighting variation. Range plots show vision falling apart at night and in rain, snow or fog, while radar keeps the same range of just under 200 meters.

Fridman frames the question as whether Elon Musk's camera bet or Waymo's lidar bet will win. He argues that fused radar, ultrasonic and camera data can do as well as lidar by some measures, at much lower cost. The open question is whether solid-state lidar will become cheap and long-range enough, but for now he says annotated driving data and learning algorithms for cameras keep improving.

On the full-autonomy side, Fridman credits Waymo with a major milestone: a public rider program in Phoenix beginning in November 2017 with no safety driver, and about four million autonomous miles, though most had a safety driver. Uber is second in miles, with about two million by December 2017. He mentions GM Cruise, nuTonomy, Optimus Ride, Voyage and Aurora as other players, and says several of their leaders will speak in the class.

On the human-centered side, he describes Tesla as the quiet player that makes no declarations of being fully autonomous. Tesla's Autopilot, released in September 2014, has over a billion miles and more than 300,000 equipped vehicles. Fridman says that in October 2016 Tesla dropped Mobileye entirely and started from scratch with mostly deep learning methods on Nvidia's DRIVE PX 2 and eight cameras, which he calls bold risk-taking that worked.

The Audi A8, due at the end of 2018, promises what is called level 3. Its traffic jam pilot works only in bumper-to-bumper traffic under 60 kilometers an hour, and Audi says that if the car was in control, the insurance process compensates victims at the company's expense. But the system can issue a takeover request with up to 10 seconds, so Fridman says it is still a human-centered system that must solve human-robot interaction.

Other human-centered systems include Volvo Pilot Assist, Mercedes Drive Pilot in the E-Class, Cadillac Super Cruise, which is geographically limited to highways, and George Hotz's comma.ai openpilot. Fridman says he is speaking so much about the human-centered side today because the class has no guest speakers for it.

Fridman lists the problems where machine learning helps: localization and mapping, scene understanding, movement planning and driver state. For localization, he describes traditional visual odometry, which tracks features from frame to frame to estimate position and orientation. Since 2015, end-to-end approaches such as DeepVO have used a CNN to extract features and an RNN to track pose over time, tested on the KITTI dataset. Because they are learnable, they improve with data.

He finds it particularly exciting that vision alone, a sensor similar to our own eyes, could be the primary sensor for controlling a vehicle. Deep learning has largely taken over scene interpretation, including detection, classification, tracking and recognition. Audio can also help, for example detecting a wet road from tire microphones, which matters because cars lack the traction information to know.

For planning, traditional approaches formalize the problem for optimization, generating thousands or millions of trajectories and scoring them against an objective function. Reinforcement learning is particularly exciting for planning and control, which is where the course's DeepTraffic and DeepCrash competitions come in. Fridman says these will be explored in the next day's lecture on deep reinforcement learning.

On driver state, body pose and head pose are well-studied problems, while finer signals such as blink rate and duration are harder. Fridman explains the "lizard-owl effect": owls move their heads to look, while most people are lizards who move their eyes. The most important and simplest driver-state task is a binary on-road or off-road glance classification, along with hands on wheel, body position and cognitive load, which his team estimates from eye-region image sequences using 3D convolutional networks.

He also notes that emotion in the car is context dependent. In their data, a smile was the feature most associated with frustrated voice-based navigation, and a stoic expression does not mean unhappiness. He closes by arguing again for the human-centered path, since localization, scene understanding and planning can be solved to about 99% with deep learning today, while the full-autonomy problems may be decades away.