Thirty thousand voltage updates a second: inside Skydio’s autonomy stack

The drone Martiros holds up on camera weighs about two pounds. Six navigation cameras sit on the airframe, each with a 200-degree field of view: three covering the upper hemisphere, three the lower, so the vehicle sees in every direction at once. More than ten deep networks run on board at the same time. At the bottom of the stack, a motor-current loop updates voltage 30,000 times a second on a chip that has roughly 200 instructions to spare per cycle. At the top, task-level decisions happen about once a second. Four orders of magnitude sit between those two clocks.

That stack is the subject of episode 42 of The Cognitive Revolution, published on July 4, 2023. Host Nathan Labenz interviews Hayk Martiros, vice president of autonomy at Skydio, the largest commercial drone maker in the United States. The company is a multi-billion-dollar unicorn whose customers include every branch of the U.S. military, and it had just raised $230 million to expand U.S. manufacturing. Labenz came in wanting the autonomy system explained from pixels to voltages, then the higher-level AI layer Skydio is stacking on top, then the uncomfortable defense questions that follow from a reconnaissance drone that still works when GPS is gone.

Martiros’s path is not the usual Bay Area one. He was born in Armenia just after the Soviet Union collapsed. At seven his family moved to Fairbanks, Alaska, where his father, a seismologist, went to study earthquakes; he spent about a decade there, then New Jersey, then Princeton for mechanical engineering. He nearly majored in music or architecture, became “the robot software guy” in the department instead, and went to Stanford for a master’s. Around 2015, with Google X buying every interesting robotics startup in sight, he found Skydio CEO Adam Bry on AngelList. Bry started asking him math questions on Skype. The “office” turned out to be a house in Atherton with the three founders working out of it.

Those three had launched Google X’s Wing drone project with their MIT advisor, then left after about a year. Robots, they had decided, cannot be a side project of a company that makes money on search ads; if you are building them, they have to be the core of the business. Martiros found that argument convincing and took the risk. He has been at Skydio for just over eight years — essentially his whole career outside school. Labenz notes a side project most listeners will know first: Riffusion, which runs Stable Diffusion on spectrograms and turns the result into music. Martiros promises not to pivot the hour to that.

Skydio began as an autonomy-software company and quickly concluded that was not enough. To have a shot at a robot people can trust, the team had to control the mechanical design, the aerospace and the vibration, the cameras and drivers and chips, and the software on top. So it became a hardware company, and for several years a consumer-hardware company — “a notoriously difficult and bad idea for a startup.” The first product was a drone smart enough to avoid obstacles, navigate without GPS, and shoot Hollywood-style footage of someone skiing or mountain biking. Tens of thousands of customers still use that. The slogan the company earned is that the drone is easy to fly and hard to crash.

The center of gravity has moved to enterprise and government: inspection, mapping, and situational awareness for fire, police, and defense. Flagship hardware flies for about half an hour with a six-kilometer radius. Skydio skipped lidar for a practical reason. On a small, light drone, size, cost, and power are everything, and the smartphone industry has already spent tens or hundreds of billions of dollars making cameras tiny, cheap, and good. The market Martiros cares about most is almost unbounded: transmission towers, bridges, dams, roads, buildings, indoor plants — every Fortune 500 company and every country is covered in assets. He had just been back in Armenia and noticed that every corner of the country is still lined with transmission towers, “just like everywhere else on Earth.”

Labenz wants the box drawn clearly: six raw video streams in, four rotor speeds out — what happens in the middle? Martiros reaches for classical control theory. If an outer loop decides ten times a second and an inner loop a hundred times a second, the outer loop should not have to care about the inner loop’s details; the dynamics can be abstracted away. From tens of thousands of hertz at the motors to about one hertz at the mission layer, that heuristic says you want something like four layers. Crossing those frequency gaps with a single end-to-end network, he says, does not make a lot of sense.

The innermost loop is motor current, at about 30 kilohertz, far from anything that looks like a language model. That rolls up to desired rotor angular rates, then to the drone body’s angular rates and acceleration, then to orientation, then to position and velocity, then to a trajectory. A quadrotor’s rotors sit mostly in plane, so it has to tilt before it can translate sideways; every layer is coupled to the one above. The core navigation layer is a nonlinear optimization, solved with second-order methods at about 500 iterations a second. Rotor aerodynamics, body attitude, obstacle constraints, and the user’s high-level command all sit in one objective, because decoupling them fails in the real world.

Picture a gust of wind, a downhill flight, and a thin power line the drone has to yaw to miss. If those objectives are solved separately, the aerodynamic demand hits the propeller’s power limit, the airframe cannot tilt, and the vehicle flies into the wire. Deep networks take the part that is hardest to write by hand. Skydio published a 2017 paper and became the first company to ship a deep-learning obstacle-avoidance system, on the Skydio 2. Easy scenes are easy. The hard cases are the ones where the pixels barely contain a signal: a textureless white ceiling versus a foggy sky of the same color, or a backlit twig in sun glare. You need the context of every surrounding camera to know whether that white is sky you can fly into or a ceiling you will hit. Deep learning crushes handwritten rules on those examples.

The company’s bet is a reliable platform with health monitoring and state machines, algorithms that keep the drone from crashing and let it know where it is without GPS, and a navigation layer that exposes APIs. On top of that API sits what Martiros calls embodied AI: models that understand camera imagery plus a high-level instruction — language, or a demonstration — and turn it into high-level motion commands. Doing that in the same loop as motor voltages is nonsense. Doing it in the same loop as obstacle avoidance is possible, not required. Labenz hears nested agent loops all the way down, each observing a state and issuing a command, spinning faster as you go inward. Martiros adds that the states you care about are rarely measured directly: angular acceleration, position, motor location are estimated, with covariances, while you command at the same time.

The library that makes the inner loops tractable is SymForce, which Martiros built as a passion project and Skydio open-sourced. It does three things: symbolic computation, code generation, and nonlinear optimization. An engineer writes a mathematical expression in Python — propeller aerodynamics, a robot-arm kinematic model — in a style close to specifying a PyTorch network, then simplifies, differentiates, and factors it without thinking about runtime. The system then emits extremely fast code. Skydio’s core motion planner compiles to 100,000 instructions, no branches, a single function, running a complicated loop in tens of microseconds on the chip in the air.

Nobody reads those 100,000 lines; the generated code is a black box in that narrow sense. The source expression is readable, testable, and iterable, and if you trust the generator you work at that layer. Martiros’s toy example is a remote-control race car: gas, brake, steering wheel in; tire-to-road contact, throttle-to-wheel time constants, and a possibly nonlinear map from ten degrees of wheel to five or twenty degrees of tire out. You write those equations, you need their derivatives, and you used to need one person who understood the theory, the numerics, and high-performance C++. Decouple the theory from the derivatives and the speed, and “any grad student that’s focused on the theory” can produce production-ready, bug-free code.

What should be learned, and what should stay physics? Martiros’s boundary is sharp. If you are sure something is true — gravity, F equals ma — do not learn it; bake it into the network’s structure or into how its outputs are used. If you are almost sure — a worn tire’s radius, drone aerodynamics in a gust, a phone-camera lens that drifts with temperature — treat the model as a strong initial guess with error bounds and let learning correct inside that range. If there is no parametric model at all, pixels to a decision, you have to learn it. The journey has been drawing those lines: geometric primitives inside a net so the job is simpler, initial guesses in the right places, post-processing into a classical optimizer.

Intermediate products are not disposable. A 3D map is not only for not crashing. It is scanned, shown to the user, used for inspection, and queried when someone taps a point on the phone and needs the exact distance. Pure end-to-end designs tend to throw those artifacts away. The minimum viable deep-learning piece, Martiros says, is recognizing that these pixels are a nearly invisible thin branch. Geometry after that — camera pose, lens model, a 2D pixel to a 3D ray, depth from motion — can be designed in several ways, more or less end-to-end. Higher in the stack, joint neural reasoning looks more and more valuable. Lower in the stack, at robot dynamics, the trade-offs still cut the other way.

Skydio’s internal name for the human-machine split is the Arc of Autonomy. Stage one: a person flies, and the machine makes it easier and safer. Stage two: the vehicle flies the whole mission; the person only sets the job. 3D scanning is the example. You mark a volume, pick a standoff — three meters, which is 1.2 millimeters per pixel — and the drone answers that it will take 40 minutes, two batteries, and 2,000 photos, then goes. The old way needed a pilot who was also a photogrammetry expert. Now an ordinary employee can watch. Stage three is the dock: a box with power and internet, installed at a substation, a well pad, a remote dam. The site is already mapped. You schedule a mission every hour or every day. The drone flies, uploads, and an API hands the pictures to whatever construction, mining, or insurance system the customer already uses.

“It has to be about as reliable as somebody installing a printer copier or buying a pickup truck or a washing machine,” Martiros says. “It just has to do the job.” The yellow vest, the hard hat, the pre-flight checklist, the trained pilot — all of that has to go away before utilities will treat a drone as a standard tool. Pricing follows the same arc. Consumer hardware is in the one-to-two-thousand-dollar range. Enterprise drones and docks run from thousands to tens of thousands, and the more end-to-end the offer, the more Skydio prices the problem solved rather than the airframe. If a drone dies, you get a new one. Two cost comparisons make the ROI visceral: a snooper truck that closes a lane and dangles a person under a bridge, where the per-minute cost of a closed lane is astronomical; and helicopters used for transmission-tower inspection, ruinous on fuel, training, and safety.

Labenz runs a back-of-envelope: a $10,000 dock, 5,000 daylight hours a year, and you are in the neighborhood of a dollar a mission — cheap enough for a soccer-practice selfie. Martiros cannot wait for that day and thinks maturity does not have to look different from renting a scooter. The path there is long: technical, product, and social. He sketches docks-as-a-service — the same box inspects a tower, beats a fire truck to a blaze, then checks a house for an insurer — and says that is further out for practical reasons. Vertical integration versus platform is a debate the company has every week. The answer is both. Language models are glue between systems; Skydio will use them and expose APIs. 3D reconstruction is a core competency, with surround cameras and data priors nobody else has, so they will keep doing it and still let customers use other pipelines.

There is no scaled dock deployment on the planet today, Martiros says — a fraction of a percent of the industry he thinks is coming. You cannot wait on an ecosystem to create that market. You have to ship killer apps yourself. Labenz’s pattern-match: in this AI wave, existing winners are in the best seat, because a challenger has to redo a decade of hardware while the incumbent stacks new models on top. On power, the compute package is maybe ten percent of a 30-minute flight; weight is the real tax. Efficiency is propeller size plus the share of mass that is battery. Five extra grams matter. Chips need cooling, cooling adds mass, mass demands more propulsion. Cars do not notice a couple of extra pounds. Every core function already runs on the drone, because a remote site may have no internet at all. Distilled language models on the next generation of chips are, in Martiros’s view, not a question — they shrink much faster than Moore’s law.

Drones, Martiros argues, are an unusually good canvas for the next wave of vision and language models. They fly close and indoors or far away; they capture rich imagery; if one crashes in testing, nobody dies, and you can iterate with paying customers already in the field. The missing piece is manipulation — they do not interact with the world the way an arm or a car does — but the combination of a state-of-the-art fleet and real industrial use is a place to apply foundation models to higher-level tasks. Labenz then asks the self-driving question. A neighbor’s Tesla on FSD had just driven him city streets to highway and off again; it felt close. Martiros is close to the problem: his best friend and several others work on Tesla Autopilot, and another friend founded Wayve in the UK, which bets on end-to-end driving.

Impressive rides are real, he says, and still a beta in which a human is expected to take over. The question that matters is whether you would sleep in the back. Would you accept a crash on one of every thousand drives? Hail and fog, a new city, a construction zone with the wrong sign — the easy case was solved a long time ago, and the distribution lives in the tail. Elon and others have said it was a year or two away for a decade. Progress is happening. Shipping a product you trust your baby to sleep in is a different bar, and that bar is higher than the human accident rate, because every driver thinks they are better than average and would rather they made the mistake than an AI. Until you are not just better than a human but orders of magnitude better, Martiros calls that product untenable. For cars, he and Labenz agree an order of magnitude is about the right hurdle.

Drones are different: lives are not on the line in the same way, and replacing a dangerous climb with a robot is an obvious safety win. The real adoption gap is procedural. A utility that already flies a manual drone has standard operating procedures, zoom-and-shoot angles, keep-out distances, a pre-flight checklist. Skydio’s vehicle will zip around, build a 3D map, and take close-ups from two feet away, which is mind-blowing and also nowhere near a process a large, slow, safety-first organization can rubber-stamp. Unpredictable motion creates worry. The product work is making the drone predictable and explaining what it is doing in the UI so someone can take action. On jobs, Martiros is more optimistic than he is about white-collar work meeting language models. The tasks Skydio replaces tend to be hot, cold, remote, laborious, and dangerous. Tower climbers and helicopter crews can become the people who design missions and run a fleet. Warehouse labor is already short. People still have to adapt, he says, or they will be in trouble — but this is not the same shock as a mid-tier software job.

Labenz is ready to endorse universal basic income; Martiros is on the Yang Gang in spirit and does not think the United States is close to executing it. “The world is barely hanging on from not just having it all be entirely authoritarian dictators.” History has digested industry after industry. The danger is every industry at once, and fast, with a real risk of extreme wealth inequality if it is not managed. The potential societal wins are enormous if the cards are played right. Labenz has taken to calling it the smartest of times and the stupidest of times, coexisting.

The largest drone company in the world is DJI, in China. Skydio is the largest in the United States, and that ranking now carries geopolitical weight. Skydio sells to local police and fire, to every branch of the U.S. government, and internationally. How directly it works with Ukrainian forces versus partners, Martiros as an engineer is not sure, but the company has sent people to Ukraine and to the Polish border to train operators. It has veterans and former Navy SEALs on staff. The published principle he cites is simple: they will not weaponize the drones. “We’re focused on the camera and the reconnaissance as the core focus of our products.” The airframe is about two pounds and is not optimized to carry much; if you want to strap something on, he says, you are better off buying a toy drone. Delivery, maybe later, perhaps with a winged vehicle — Zipline is the American company he points to — is not the focus now.

In Ukraine you can assume GPS does not work. It is jammed or spoofed. That is exactly where Skydio’s visual navigation is supposed to shine. Small drones are orders of magnitude cheaper than human lives or traditional defense equipment, and that exchange rate is rewriting the battlefield. Labenz cites a report that Ukraine is losing about 10,000 drones a month, mostly to electronic warfare. The product feedback that comes back, Martiros says, is first of all less reliance on external signals. The average robotics engineer is not in that mindset. The execution risk for the company is spreading across too many good use cases and never making any of them amazing — the same horizontal-versus-vertical tension as the dock. Ukraine matters, and Skydio is not a defense contractor, and that cannot be the primary focus either.

The conversation ends closer to home. Martiros had just been at the Azerbaijani border, flags of both sides lined up, and the day after he left the area there was a drone strike. Turkish drones going to Azerbaijan are part of that conflict. For a bunch of reasons, he says, he is against weaponized autonomous systems, a stance he calls common among robotics researchers and AI people, and hard to hold when the arms-race logic says the other side is doing it so you must too. “Any sort of powerful new technology, it gets used in a lot of really good ways and it gets used in some really bad ways.” Labenz, looking back across the show, notices that nearly half the guests grew up outside the United States — Jordan, Russia, France, Egypt, India, Germany, Israel, China, and today Armenia — and hopes the country keeps that open posture. A drone’s layered architecture can be taken apart. So can the talent mix of the industry that builds it.