Why Vijay Kumar Refuses to Call Them Drones

When Lex Fridman asks Vijay Kumar about the first robot he helped build, the answer is not a tiny quadrotor. In graduate school Kumar worked on a hexapod that weighed close to 7,000 pounds and walked on hydraulic power. Eighteen hydraulic motors each ran on their own Intel 8085 or 8086 processor, and a nineteenth computer coordinated the joints. His thesis asked how to orchestrate those eighteen legs — especially the pressures in the cylinders — so the machine could locomote efficiently.

Clock speeds were about half a megahertz, and messages still had to cross from joint to joint hundreds of times a second. Kumar says the people on that project wrote what he believes was one of the first multiprocessor operating systems, in the 1980s. Seeing the monster move was amazing, he recalls, though in hindsight the whole thing should have been much smaller. Today's machines from Boston Dynamics and from Ghost Robotics, a University of Pennsylvania spinoff, show how far the substrate has shrunk.

Back then, compute and sensors forced everything to be unnecessarily large. Human psychology still treats bigger as grander: people never grant nanotechnology the awe they give a space shuttle or a Boeing 747. Kumar's later career is an argument in the other direction. Small, he has spent decades showing, can be the more beautiful machine.

Asked for the most elegant robot motion he has seen, Kumar does not pick a single favorite machine. He is proudest of small vehicles that maneuver in tight spaces and coordinate to form three-dimensional patterns. Once they can do that, they can create objects in the sky and reshape them on the fly. The lab had already done the two-dimensional version with ground robots; hanging the same idea in midair was a different accomplishment. In 2011 it was news. Eight years later, he notes, it looks almost standard.

Beauty to an engineer, he says, is not the same as beauty to someone watching from outside. What he means is physical form: getting a shape in midair and letting it deform. The agility of each vehicle matters, but distributed cooperation is where the beauty emerges. Fridman then asks what to call the machines — UAVs, quadcopters, drones.

Kumar does not like the word drone. To him it is pejorative: something dumb and preprogrammed that does one little thing. Robots, he argues, are anything but that. Unpiloted is not quite right either, because a radio-controlled aircraft is unpiloted too. The accurate phrase is aerial robot, preferably with two adjectives in front of it: agile and autonomous. Kumar is a professor at Penn, dean of Penn Engineering, and former director of the GRASP Laboratory — General Robotics, Automation, Sensing and Perception — founded in 1979, older than many of the engineers now building flying machines.

The living systems that instruct his engineering are ants. Individuals are structurally simple, yet robust: take away one of six legs and the ant still walks, possibly without noticing. Colonies add a second layer. They chain themselves into a raft in a flood, recruit teammates to carry large food, and forage in different directions until they reach consensus — without talking the way humans talk. “In some sense, they also know how to do democracy probably better than what we do,” Kumar says.

Fridman asks whether a swarm should be treated as one living organism, the way one might treat human civilization as a single creature. Kumar's answer is an engineer's. The point of the discipline is to go beyond individual units and treat the group as a cohesive whole. Obsess over every building block and the math explodes: ten agents means a ten-fold Cartesian product, and design in that high-dimensional space is nearly meaningless.

The right move is to design individuals so that, at a higher level, they collapse into a low-dimensional abstraction. That means two concrete choices: make each agent's view of the world as local as possible, and make each agent robust to collisions, so one failure does not kill the swarm. Engineers already do this with airplanes and iPhones — well-specified interfaces, predictable components. Swarms add a further demand. The population itself must be resilient: able to re-establish communication with neighbors, rethink group strategy, and reorganize. That, Kumar says, is where a lot of the hard problems sit.

How much information does a flying swarm need, and can it form without a central controller? If you are a purist, Kumar says, you try to recreate nature. Organisms group for shelter, food, and procreation. Interactions are organic and local; shared information is mostly indirect, aimed at keeping the colony alive. Engineers, by contrast, have a mission, and missions that involve mobility are almost always stated in a global coordinate frame.

“Surround this building and protect it from intruders” is a building-centered command. Someone has to tell the robots where the building is, and they have to collaborate on a map of it. That information is built up explicitly, whereas nature lets structure emerge. Nature is sophisticated, Kumar says, but the tasks it solves are very different from the artificial tasks engineers are forced to address. You are driven into a setting that is, in his word, unnatural — not how nature does it.

Nature is also less sensitive to losing individuals. Robotics still is sensitive, because robots are expensive. The caveat is economic: the price-to-performance ratio of robotic components is falling dramatically, and Kumar thinks the field is asymptotically approaching a point where the cost of an individual becomes insignificant. Swarm tactics that tolerate sacrifice only look like engineering, rather than a laboratory game, if that cost curve keeps falling.

Kumar sketches the landscape of autonomous flying vehicles. Military aircraft get the most attention: they are run from a base station with heavy human supervision and limited autonomy, mostly going from A to B. More sophisticated types can take off and land on their own; they are usually winged and heavy. Military organizations already field autonomous helicopters, and commercial autopilots are a close cousin. He even asks how many air-safety crashes would have happened if the plane had been truly autonomous — many, he suspects, come from pilot error, because people make silly decisions.

In the lab they can be more aggressive. The question they try to ask is whether a robot can decide without any external infrastructure. The most common crutch is GPS, which Kumar calls a brutal form of information: drive among tall buildings and it drops out. The second is communications, which are easy to jam; Wi-Fi and cell signals drop routinely. The third, which he hates to call infrastructure, is people — someone on the other end of a radio who can pilot you.

Strip all three: no pilot, no link to a base station, no knowledge of position, no prior map, no model of what the environment will do next. Can the robot still navigate? That, he says, is true autonomy. Agility belongs in that picture because robots should work in constrained places a Global Hawk never sees: inside a building for search and rescue or an active shooter, under orchard canopy to count fruit and measure trunks — work his group has actually done in agriculture. Agility is not a hundred-meter record. It is seeing the unexpected and maneuvering safely in a way that gathers the information the mission needs.

Kumar may be the only person to put a math equation in a TED talk. Curator Chris Anderson told him not to; he did it anyway, because that is the work and the audience deserved a taste. The practical question underneath the formula is more basic: what does it take to make four motors fly? His group had been trying since 2000 and could not, in part because sensors and computers were not ready. Between 2000 and 2007 several things arrived at once — Moore's law, the iPhone, the cloud — and inertial measurement units matured.

DARPA had poured money into IMUs without intending a boom in small aircraft. What actually crushed accelerometer cost was a regulation: every car needed an airbag, and every airbag needed an accelerometer. Federal spending on one problem accidentally made small flying robots possible. Kumar notes he could never have raised that much money by asking for low-cost IMUs directly. That, to him, is the interesting part of research: outcomes are extremely hard to predict.

By 2007 and 2008 they could build the vehicles. Today a high-school student with a Raspberry Pi kit can get the low-level loops running because so much is commoditized. The physics is unchanged: drive motors to the right RPM, produce the right thrust, place and orient the body. Feedback comes from onboard sensors. The IMU reports acceleration and angular velocity; you still need local position or velocity, the way a walking human fuses stride length, optic flow on the retina, and inner-ear accelerometers. Robots need a camera or similar speed sensor, plus a global frame if they must act in world coordinates.

The configuration is underactuated: six degrees of freedom, four inputs. Sideways flight is not a mystery, and yaw comes from spinning the rotors at different speeds to generate reaction moments. From an engineering standpoint it is simple, clever, and versatile. The four-rotor idea is about a hundred years old; Kumar's lab first tried eight rotors on a sphere for omnidirectional flight and ran into thrust-to-weight and sensing limits. People who knew rotorcraft already knew four would work. The agile indoor machines only became possible in their lab between 2007 and 2009, once onboard sensing and compute caught up.

Getting from A to B starts with a trajectory. Planning, Kumar says, used to sound esoteric; self-driving cars made it ordinary. A vehicle that sees a scene, holds a lane, or turns right has to plan a path that is safe and efficient. Safety cannot be compromised. Optimality can: there is a time budget, so global optima become good enough. Efficiency might mean fastest, most graceful, or least energy, always inside the safety envelope. The lab focuses on smooth trajectories that satisfy those constraints.

Fridman asks whether deep learning and reinforcement learning belong in flying robots, noting that most flying success so far is not machine learning except in vision. Kumar pushes back: they rarely labeled the work “learning,” but they have used it for years. The first attempt to fly through a constrained space almost always fails, because perfect execution would require modeling everything in the environment, and aerodynamics is notoriously hard. Just before the interview he was discussing blade flap with a student: models treat blades as rigid, but they bend, and at high speed the effect on angular acceleration is significant.

Near the ground, the vehicle is pushed away — ground effect, which every pilot knows. Near a ceiling it is sucked upward, which almost no one thinks about. Walls produce wall effect; two trains passing buffet each other. These microclimates are not impossible to model, but the required fidelity would be tremendous and tedious. So the group adapts over time. Early on they used iterative learning: tweak a few parameters across attempts until the task works. You could call it a policy gradient or reinforcement learning; it is iterative learning.

In today's autonomous vehicles, learning can sit in perception or in action. Everything Kumar has seen succeed in the field is on the perception side: detection, classification, tagging — a decade of computer-vision progress. On the action side, he does not know of a fielded system that learns the right behavior beyond a single demonstration; laboratory results do not count. End-to-end, pixels to motor currents, is the holy grail and “really, really hard.” Going forward, data-driven methods should work in concert with model-based ones, not replace them.

Fridman presses on the limits of vision-only perception, citing Elon Musk's bet on cameras. Kumar's spinoff, Exyn Technologies, works underground in mines: dark, dirty, propeller downwash kicking up dust. He challenges anyone to run a computer-vision algorithm there; they use lidar. Indoors and in fields, he thinks vision alone has real potential. The deeper question is whether a single sensing modality plus learning can identify every corner case.

Hit a corner case and the instinct is to collect data and patch it. At some point that saturates. Vision systems might detect objects 90 percent of the time, and cats on the internet maybe 95 percent. Going from 90 to 99 percent takes much more data. For a safety-critical application, 99 percent is not enough; 99.9 percent takes even more. Plot accuracy on the x-axis and data volume on the y-axis, and Kumar believes the curve is exponential. Even a linear curve would be painful.

There is a second axis: the process is extremely power-hungry, and he means electrical power, not a metaphor. In 2014, data centers used 2 percent of U.S. electricity consumption — a five-year-old figure even at the time of the conversation, but enough to make the point. People treat this as an information-processing problem; it is an energy-processing problem. Without more efficient methods, he does not think the path is viable.

Is autonomous flight harder than autonomous driving? Flight has a remarkably safe default: climb vertically to a maximum altitude, fly horizontally to the destination, descend vertically — a trajectory you can preprogram. Ground vehicles have no clean equivalent. They move on a two-dimensional surface where paths are more likely to hit obstacles, a fact that is both intuitive and mathematical. The downside of flight is that you must model and work in a three-dimensional world, which is significantly harder, and you must live with the peculiarities of aerodynamics.

Delivery is the application he is bullish on. In a city like Hong Kong, crossing a river can take half an hour; a flying vehicle can do it in five minutes. With the nonprofit WeRobotics in the Peruvian Amazon, rivers are the highways: five hours by boat, thirty minutes by air, for drugs outbound and samples back for vaccine testing. The bottleneck, he argues, is economics more than technology.

Almost nobody dwells on the constraint that actually bites: battery technology has not had a huge breakthrough. Packs are cheaper because of megafactories, but they are still lithium. Two numbers are fundamental: energy density and power density. Power matters because most of these vehicles take off vertically with no runway. At small scale, vertical takeoff consumes roughly 200 watts per kilogram. The entire human brain uses less than 80 watts. Lifting yourself into the air, he says, is like turning on two or three light bulbs, which “makes no sense.” Power decides whether you can leave the ground; stored energy decides how far you go.

Companies promising flying cars that are autonomous, electric, and clean are overpromising on the clean part. Autonomy is doable; electric is not, except for short hops. Another company he works with builds small jet engines that easily reach 50 miles per hour and lift 50 kilograms — efficient, louder than electrics, and immediately viable if you accept fossil fuel. He would not bet against people like Sebastian Thrun, and he does not think flying cars have to be rotorcraft: take off vertically, then morph into forward flight. The real question is economic viability.

Many roboticists treat people as a mess to be designed out — too hard to model, a source of uncertainty. Kumar rejects that. He does not think we should ever think about robots without a human in the picture. Robots exist to solve problems for humans, but there is no general theory. He splits interaction into three modes: a human commanding a robot; true collaboration, such as a robot and a person lifting and carrying together; and humans as bystanders, the self-driving-car case of merely acknowledging that people are there. Different tasks pick different modes. Treating robots as independent of humans is, in his view, equally silly. Human-robot interaction is almost a mandatory piece of everything they do.

The Level 2 versus Level 4 debate makes the handoff problem concrete. Relinquishing control to a machine and taking it back when needed is harder than it sounds; Kumar has felt that on the highway. Tesla's approach is interesting because it forces periodic contact with the car. Toyota talks about shared or collaborative autonomy. Both, he argues, are still simple forms of collaboration, because sitting in a vehicle from A to B is a boring task. Search and rescue is the interesting one: human and robot first responders, a burning or collapsed building, two minutes on the clock, a highly unstructured scene.

He also flips the modeling question. Drivers constantly estimate the state of the car and the world; what if the car estimated the state of the driver? Phones already guess what you are doing and push reminders. A car that watches eyes and steering could make the driver safer. That is human modeling, not robot modeling, and he thinks robots can do it well: a person in a cockpit ringed with sensors, more sensors outside, a two-way stare. Fridman calls it a new, AI-enabled psychology.

Fridman asks about public fear of walking machines like Boston Dynamics' and of flying ones like Kumar's. Kumar does not dismiss it. Any technology meant for good can be subverted; robotics is no exception, and robots are easy to weaponize. Getting a swarm to do something reliably is extremely hard. If the goal is destruction and only one member of the swarm needs to arrive, the problem becomes much more doable. That gap — full success versus one success — is what worries him about autonomy at large scale.

If someone is going to develop the technology, he would rather the good guys do it, so they understand how it can be abused and how to defend against it. His group studies defense against swarms as well as the swarms themselves. The National Academies have a report on counter-UAS technologies; he treats the threat as real. Knowing how swarms and autonomy work is a prerequisite for defense. Engineers belong in that discussion. “The days where politicians can be agnostic to technology are gone.” He calls technology the new liberal art: everyone, not only experts, should understand how it will change their lives. He adds, half joking, that we might elect a few engineers.

The biggest open problem, as he restates it, is not a single algorithm. Robots today are still tailored to specific tasks and settings. How they operate in broader, changing, unstructured environments is unsolved. You can do Level 5 autonomy in a parking lot. Can you do it in the streets of Naples or Mumbai? No. Robotics has to talk about where the machine works, what the environment is, and what the task is — and the field has no systematic way to put those three together. Board games do not prove we understand intelligence: they are as structured as a Henry Ford line. Structured settings are the easy ones, and we already know how to do them.

For a first-year student interested in robotics or engineering, Kumar offers three pieces of advice. First, get used to the idea that the world will not be the same when you graduate in four or five years. Predicting the future is hard, and you will not be good at it, but the habit sharpens your senses and makes you smarter. Second, a corollary: you do not know what will matter, so specializing for a single path is not enough. Breadth gives you a jumping-off point when the important problem changes.

Third, he credits Penn with teaching engineering inside the liberal arts and inside society — a perspective engineers cannot afford to lose. One thing he thinks people underestimate is mathematical foundations and the importance of representation. Not every problem yields to a ROS package off the internet or a deep network that happens to work. Representation is key even in machine learning: if you hope for explainable AI, you will need representations a human can understand. “If you want to do robotics, you should also do mathematics.” Fridman asks whether that includes Dostoyevsky. Kumar agrees.