Kyle Vogt Built the Highway Kit, Then Scrapped It Before Selling a Single One

Lex Fridman introduces Kyle Vogt as president and CTO of Cruise Automation, trying to solve vehicle autonomy, and as cofounder of two companies — Cruise and Twitch — each sold for a billion dollars. The conversation sits inside MIT’s artificial general intelligence series. Vogt grew up in Kansas City. The FIRST Robotics team that would take him was not even at his own school. Shawnee Mission North High School had the only teacher in the area willing to sponsor one, so he went there.

The snowball started with Legos. As a small child he wanted the kits with motors; then Mindstorms arrived and, for the first time, you could program the contraptions. BattleBots on Comedy Central finished the job. Two-hundred-pound steel machines taking each other apart in a transparent box looked like the coolest thing in the world, and he had no idea how to build one. He did what a kid in that era could do: he read, by his estimate, tens of thousands of forum posts from the people who actually competed, triangulated a method, and at 13 or 14 showed up with a robot. They lost every first-round match. Autonomy was allowed in some limited form, but in a two-minute radio-controlled bout, driving it yourself was the practical move.

The engineering lesson was at the limits. They bought $40 DC motors off the winch on the front of a pickup, then ran them at double or triple rated voltage. The motors started overheating immediately. For those two minutes you could pull a significant increase in power before they burned out. They did the same thing to battery packs and to every material in the system. There was something intrinsically interesting, Vogt says, about seeing where things break. The first robot was a boring, extremely effective wedge: get the front edge under the opponent and they flip like a doorstop. A pneumatic polished stainless-steel spike shot out about eight inches and was, he admits, pretty ineffective — it just looked cool. Eighteen months later came a titanium axe with a hardened-steel tip, driven by a hydraulic cylinder pressurized with liquid CO2, which created its own set of problems.

Software arrived on Apple IIs, first as a crash that dumped him to a prompt his teacher cleared with a command he still remembers, then as numbered BASIC in fifth grade, leaving gaps between line 10 and line 20 so you could insert a 15. High school had no computer-science classes; he taught himself. The turn toward cars was a drive, not a match. Newly permitted, he spent ten mind-numbingly straight hours across western Kansas toward a Las Vegas BattleBots event and thought, even with mediocre programming, that a computer could do this: take a picture of the road, find the yellow lane markers, steer the wheel. He later learned people had done versions of it since the 1970s or 1980s. He still wanted to do it. After that trip he left radio-controlled fighting machines for autonomous vehicles, starting small and electric.

Before convolutional nets, his picture of perception was heuristics. Extracting yellow lane markers from a road image is the sort of problem old-school image processing can actually do: threshold on yellow, fit lines with a Hough transform, go from there. Cruise’s first days used the same move — a human-written color threshold to stand the scaffolding up before migrating to deep learning. His first system ran on a 233 MHz Pentium, first version in interpreted BASIC, the only language he knew. There was no compute for the nets kids email him about now, following TensorFlow tutorials at 11 or 12. He can only imagine a 13-year-old starting in Python and ending up after 10 or 15 years. Fridman notes that end-to-end optimism is the kind of thinking you do not want to kill, because eventually it will be right.

He had always wanted to start a company — high-school servo-driver boards sold online — but felt he had the environment, the people, and the free time only once he reached MIT as an undergrad in 2004, the year of the first DARPA Grand Challenge. There was no official faculty-sponsored entry. A bunch of undergrads, mostly freshmen and sophomores, haggling sponsorships, a donated vehicle, a pile of sensors, lined up against postdoc- and faculty-led teams. They got drive-by-wire and very basic control running. On the day of pre-qualifying, the one and only steering motor they had bought — the thing retrofitted to turn the truck’s wheel — died. The vehicle sat dead in the water. They did not make it very far.

The spirit of that challenge, as he remembers it, was curiosity: is this even possible? The million-dollar prize pool is, dollar for dollar, one of the most effective uses of taxpayer money he has seen, the catalyst and tipping point for the modern wave. Fridman asks whether DARPA should run another, aimed at pedestrians and messy cities. Vogt’s instinct is no. Spinning up the flywheel was the job, and it was cheap. Today the problem is commercially interesting to very large companies; the money and the people in classrooms are orders of magnitude beyond 2004. They can say mission accomplished and move on to the next field that needs help.

Junior year he left. Two guys from Yale were starting something. He googled them, found they had sold a previous company on eBay for about a quarter million dollars, and decided they were rock-star entrepreneurs who must be driving Ferraris. He flew to California on a one-way ticket for MIT’s January independent-activities month, planning to stay a month. He basically never went back. They were having too much fun building. The company was Justin.tv: a reality show about a guy named Justin, then a live video platform, then Twitch. It did not feel like a brave burn-the-ships decision in the moment. A month was low risk; a semester off was something MIT would tolerate; after two or three cycles he threw in the towel. He felt he could always hit the undo button. At the time the flux out of MIT was into finance and consulting in Boston or New York. Very few people went to California to start companies. Today, he says, that is probably inverted.

Fridman has a story from midnight on March 18, 2007: TechCrunch ran early, the site was not ready, Vogt and Emmett Shear coded through the night. Justin Kan, in an article Fridman read, had called Vogt a creative genius known for mulling through problems. Vogt would have been about 21 and not very experienced. In a startup you are racing the clock. The plan was to launch the second the live-streaming camera backpack worked — Justin could wear it anywhere in the city, before the iPhone, and the video would go out. They thought they were there, sent the launch emails, then realized they were not. They told themselves the announcement would wait until 10 a.m. It was 5 p.m. Seventeen hours.

The system was messy. To get a decent live feed from anywhere in San Francisco he bonded three or four cell-phone data modems, sprayed the video stream across them, and tried to catch every packet on the other side of unreliable cellular networks. On top of that sat homemade protocols for reordering, time buffers, and error correction. The night before launch, every five to ten minutes the picture went to static and the audio, also corrupted, produced a horrible screech. It was off-putting to viewers. Four people in the company; two wrote code. Shear did the website and chat. Vogt owned the device and the video server, alone. "It's my sole responsibility to figure that out." Deadlines and intense pressure, he says, are when people sometimes do their best and most interesting work. It was a terrible moment. He looks back on it fondly as a character-defining one.

Cruise Automation was founded in October 2013. Twitch was successful, and the work was entertainment: a better product meant more people watching, more ad revenue. It felt good, and it was not enough. He wanted something that scratched an existential itch. The constraint in the back of his head was that Twitch had taken about eight years, so whatever came next he had better be willing to give at least ten. That timescale raises the bar by itself. He wrote three requirements. The technology itself had to determine whether the product succeeded — juicy, hard problems, because that is what motivates him. It had to have a direct positive impact on society: healthcare, self-driving cars that save lives, a clear line to other people’s lives. And it had to be a big business, because impact only counts at scale.

He tried a Gmail clone and poked at other ideas. Then the light bulb: self-driving cars, the most fun he had had in college. Ten years had passed since the Pentium. He looked around and found almost nobody except the Google project. An entrepreneur’s job, he thought, was to find the minimum viable product. He called it probably the greatest applied AI problem of the generation; if it worked it would be both a huge business and the most positive impact he could have. Google had poured money and still had not cracked a fully driverless car. His path from zero to scale was a simpler product that created value: highway driving, with a backup driver, sold as a retrofit, profits recycled into fully driverless research.

The only thing that really changed is they never launched the first product. After about a year the highway autopilot worked at prototype. Investors had shown enough interest and enough of a signal. They abandoned it completely and went all in on driverless cars because they could not think of anything more exciting or, if it worked, more impactful. Fridman still likes the retrofit as a scale idea. Vogt has come full circle: not at all. Controlling steering and other inputs is safety-critical and hard to get right aftermarket. Even if you did, liability explodes when the base vehicle fails, when ABS kicks in, when the manufacturer silently ships a software version you did not validate. The long tail is basically infinite. For a safety-critical product that is not acceptable. He did not know any of that at the time, so they tried anyway.

The business-model hole was almost bigger. They started on an Audi S4 and assumed three makes and models would cover 80 percent of San Francisco — doesn’t everyone drive a BMW or a Civic? The user survey was all over the place. A decent number of units meant supporting 20 or 50 models, each a little butterfly of custom hardware and integration that takes time to maintain. A tough business, he says, and that was before sitting inside an OEM.

GM manufactures and sells more than nine million cars a year. Closing the gap between Detroit and Silicon Valley starts, Vogt says, with an acknowledgment: the automaker is tens of thousands of brilliant, motivated people who want to be part of the future. The attitude inside is to embrace the transformation rather than fear it, a testament to GM leadership that reaches the people on the plants. Shared values — we want this to work because it is important — make the rest possible. The cultures still clash. In a car company, promotion and the reward system are about following process and delivering the program on time and on budget. Risk-taking is discouraged because a late program or a plant down for a day burns millions you can count. At Cruise around the acquisition, the reward structure was solving messy problems any way you could, including crazy ideas of which 90 percent would not work.

Meshing continuous improvement with a world where everything must be rigorously defined up front, so you never slip a deadline or miss a budget, was a pretty big challenge. More than three years after the deal, the investment in who does what, and in bridging two engineering systems, has become one of their greatest assets. For a while both GM and Cruise were very steep on the learning curve. Fridman offers lawyers who say most of their work could be automated except that the infrastructure is still paper, and notes that nobody in Silicon Valley ever hits a deadline. Vogt thinks that last part is probably true. Automotive at least is fundamentally software, which means hiring software engineers and thinking in a software world — if you can close the culture gap.

Silicon Valley, he warns, is too ready to write off building cars because the country has been doing it for over a hundred years. Solved is not the same as easy. The mistake is to think Silicon Valley engineers can solve any problem, therefore a car is not a real engineering challenge. After seeing the scale and industrialization inside an assembly plant, he is very glad they do not have to reinvent it. Partners with a century of process, infrastructure, and supply base are remarkable because the surface area of deploying driverless fleets is so large that Cruise is constantly looking for ways to do less and focus on what matters. If they also had to develop, assemble, test, and build the cars in-house, the problem would become intractable. Fridman ties it back to the DARPA motor: it would have been nice if someone who knew motors had handled motors, so the undergrads could write software.

Fridman asks where the money is. Vogt answers with a short model. The economics of a driverless fleet hang on a handful of variables: the cost to build the vehicle, sensors included; the lifetime — 100,000 miles versus two million is a different business; utilization, like an airline, as many revenue hours a day as possible once the asset exists; and how you actually generate revenue. Ride-sharing is the obvious tap, with existing demand in large urban markets, plus real product benefits versus today’s Uber: privacy, consistency, and, he hopes, significantly better safety. It is also crowded. Delivery — parcels, food, groceries — has heated up in the last six to twelve months. Once you have the core fleet, many businesses can sit on top. Until you have cars as good as or better than humans, there is zero monetization opportunity. That is the holding pattern for the entire industry.

Could a rider pick a driving personality — more aggressive, more conservative — the way you cannot pick an Uber driver’s temperament? Within reasonable bounds, Vogt says, yes. They are highly unlikely to expose any knob that significantly increases safety risk. Relaxing comfort constraints slightly is plausible. He files all of that under nice optimizations after the core problem is solved and the fleets are out. The intuition that flooring it when the light turns green gets you there faster is mostly self-deception. On a 15-minute trip, even if the autonomous car does not put the pedal to the floor, you are within 30 seconds of the aggressive human. Fridman is less interested in the seconds than in the emotional release: a car as the real-world version of being a troll, a shell from which to yell. Autonomy would have to give people some other outlet.

Vogt has a comparison he likes. He has read that Americans spend over an hour a day on social media, staring at Facebook: 3,600 seconds, probably not productive, that you do not get back. On the road, another vehicle — human or autonomous — delaying them by three seconds gets the horn, one-thousandth of the daily Facebook waste. If everyone is in a self-driving car, reading or working or talking, do they even notice the three seconds? Unknown, and worth watching. At scale, he suggests, self-driving cars will lower society’s collective blood pressure by a couple of points. Less road rage is a positive externality.

What is actually hard in San Francisco? He has a hard time answering because they have been on driverless cars for a little over four and a half years, and even a year or two in he felt the functionality was there: left turns, around a double-parked car into oncoming traffic, through construction. The scaffolding was early. The benchmark is human driving, and humans are excellent at edge cases and unexpected scenarios, which is exactly where computers are not. Beating that baseline is the work. They find the most uncomfortable, safety-relevant events, categorize them, rework a piece of the system, and repeat. Overall performance climbs at a pretty steady clip. There is no one thing. There are thousands of little things: polish, every permutation, more deep-learning modules, more test coverage, more scenarios. "We're sort of in the unsexy phase of development right now" — the engineering that takes a prototype to production. Fridman calls it scaling the grinding. Vogt does not think the grinding ever stops. You cross a human-performance threshold and become superhuman, but there is no first-principles reason capability taps out near humans. It could be 20 times better, or a thousand, given time. They intend to chase that forever.

When will major cities have hundreds of thousands of fully autonomous vehicles? Fridman offers Rodney Brooks at 2050 and beyond, and Elon Musk saying it should have happened two years ago. Vogt would have liked two years ago; they are not there. He flips the question: what would prevent you from getting there? The consensus is to start in easier operating domains — less weather, mostly sun — and add capability. That caps how many cities you can enter. When heavy rain and snow fall away, the market might open two- or threefold. Production is not the bottleneck. If they wanted to, they could produce that many cars today; they could not use them, because they would saturate demand in the first cities. The real clock is those restrictions disappearing. His guess: less than five years.

Two billion-dollar acquisitions make him a reasonable person to ask what a startup takes, with the survivor-bias caveat. Three common threads. First, real passion for the core technology — lying awake thinking about the problems — because the emotional weather of a company is violent, on top of the world one day and over tomorrow, and unpaid love is what you fall back on. Second, the people: cofounders who are logical, always pushing their limits, with very high integrity. At Cruise that is Dan Kan; at Justin.tv and Twitch it was Dan’s brother and a couple of others. Third, persistence: conviction in the original premise, plus willingness to do the unsexy work, finance and HR and operations, grinding toward a North Star a little every day. The only things that can slow you down are running out of money or competitors destroying you. Most of the time, he thinks, it is founders giving up or wrecking things themselves. Fridman’s shorter version: if you never quit, eventually you arrive. Vogt says that is better.

He went through Y Combinator twice. There is no one-size-fits-all path, but if the ambition is to see how big something can get, the venture-backed route keeps capital from becoming the primary constraint. He loves YC less for the curriculum than for the cage: the top maybe 1% of highly motivated peers at the same stage, an environment that breeds success because you want to emulate them or beat them. The second time, he already believed he was self-motivated. The problem was going to be hard. Surrounding himself with other founders would make him push a little harder. That was enough to make it worth doing again.

If he could go back, he has screwed up pretty much everything at least once and does not regret those, because even the bad outcomes taught him something. The closest thing to regret is a stretch at Justin.tv after about seven years, when the company was becoming Twitch and video gaming. He is not a gamer and barely uses Twitch. He was still on the core technology; his heart was not in it. It did not meet his bar of existential impact. He probably spent an extra year or two. He would tell that version of himself: you are not getting any personal meaning out of the work, find a way to change that. It is now part of the pitch to everyone who joins Cruise — you have that here. Fridman wonders whether those two years of dread were the fire that made Cruise. Vogt says it is a good theory. 2019, for Cruise, is prototype to production: try to cross a superhuman software threshold and get the thousands of other little building blocks in place to launch a first commercial product. A lot of work, a lot of brilliant people, including names Fridman knows from the security world, Charlie Miller and Chris Valasek. It is all up to them now.