From Sandwich Math to Expert Data: Scale AI's CEO on What Actually Makes Models Smarter

Droege's first company, Scour, was co-founded with Travis Kalanick when both were around 19. They built a multimedia search engine and peer-to-peer file-sharing network from a UCLA dorm room, initially hosted under a subdomain of the computer science department's servers. The department turned out to be pleased about it rather than angry.

The financing taught him his central lesson. As the deal with the first investors progressed, the terms kept shifting: a few million dollars at a $5 million valuation turned into demands for 50%, then 75%, then 80% of the company, with pressure to sign that day. Droege concluded that "there is no way to do things", only the way you negotiate through the world, and says this shaped both Kalanick and himself at Uber.

Scour was eventually sued for roughly a quarter of a trillion dollars by the entertainment industry's trade associations, then settled for a million. To Droege that was a cold splash of water: established companies had no playbook and simply made up numbers to drive a small competitor into bankruptcy. If something that devastating is going to happen, he jokes, doing it at 21 or 22 is the time to do it.

In his first interview since becoming CEO, Droege clears up the Meta deal. Meta invested a little over $14 billion for 49% of the company in non-voting stock, Alex Wang fills the one board seat tied to it, and governance is largely unchanged. There is no preferential access or relationship, privacy and data-security controls remain, and only about 15 people went to Meta. He says Scale now has two businesses, each with hundreds of millions in revenue, and that it has grown every month since the deal.

On the market itself, Droege dismisses competitors' claim that Scale is stuck in low-cost generalist labeling as "bogus". He traces the company back to 2016, when Wang realized that data was the key input to models, starting with autonomous vehicles, then computer vision and defense work, and then the generative AI wave. As models improved, the data they needed changed, and Scale changed with them.

Eighteen months earlier, a typical task was judging whether one short story was better than another. Now a single task might be building a full website by one of the world's best web developers, or explaining a nuanced cancer topic, and takes hours of expert time. According to Droege, 80% of Scale's expert network has a bachelor's degree or higher and about 15% hold PhDs. The best experts come mostly through referrals and campus programs, because they take part partly for the money and partly because they care about improving the models.

Asked whether the economy will end up as mostly people training AI through reinforcement learning, Droege says RL matters and ties it to the broader move to environments: sandboxes in which an agent practices reaching a goal. His example is a Salesforce instance full of data and configurations, where an agent must run a business process with high reliability and know when to hand off to a human. Scale has done this work for over a year.

The hard question is generalization. Software systems vary in configurations, data types, sizes and user counts, so the permutations are endless, and labs cannot collect data for every combination. The more broadly a task transfers, such as finding a meeting on a calendar, the more valuable the data. Scale's job, he says, is to supply the data that makes agents most useful across many end-user situations.

On what the human contribution looks like, Droege says it can be a finished website or an annotated one explaining why each decision was made. It can also be a broken site with an explanation of what is wrong, if the goal is a debugging tool. What gets collected depends on what the model builder is trying to teach.

Scale's second business builds applications and services for healthcare, insurance and government. Droege describes a hospital system whose specialists see rare cases and must read 200 to 300 pages of mixed-format records for each patient. Scale built a tool that reads the file and surfaces the five to 10 most important considerations, in one case catching an allergy that conflicted with the planned medication.

To improve such a tool beyond what off-the-shelf models, retrieval and fine-tuning can do, the hospital's own people have to label data. Droege says labeling is therefore moving inside enterprises and governments. A bank may ingest hundreds of petabytes a year, but most of it is not useful; the scarce thing is how a specific banker or doctor would decide given the company's culture, objectives and incentives. A document with the same words can mean something different at a different company.

He frames this against two extreme narratives, imminent AGI and "it's all a bubble", and says the truth sits between them. Mission-critical agentic systems still struggle to reach the accuracy needed, and each organization's context has to be built in.

Asked how long models will need human input, Droege notes that data labeling's history is one of new beginnings: autonomous vehicles need less labeling than before, but new needs keep appearing. For models to stop needing outside human data, he argues, no new human skill or knowledge would have to be worth adding, which seems far-fetched. He admits Scale is financially motivated to say humans will remain involved, but says he also believes systems should work for us and keep us in the loop.

On the "white-collar apocalypse" he calls himself practical. The transformation might happen, but not in the next year, and two years is far-fetched. Humans have adapted to every earlier technology scare, he says, and the doom narratives underestimate that.

A lot of what experts produce is evaluation data, especially for enterprise and government customers: someone has to set the benchmark for what good looks like. In the healthcare case, a doctor would define what a model should find in a record. Droege adds that AI shines where human processes are 10 to 20% accurate and AI brings them to 70 or 80%, but a process already at 98% is much harder to improve.

Lenny observes that models are no longer simply fed all of human text; they improve as people fill specific gaps. Droege compares it to laying broadband: someone has to dig up every road. He says model progress is a combination of compute, model improvement and data, all improving at once.

Droege says the general trend is a move from models knowing things to models doing things. Benchmarks suggest model knowledge is now quite robust, so the question becomes what a model can do for you, such as navigating a Salesforce instance, a healthcare system or a weather app, and how an agent makes decisions on your behalf. People disagree widely on the pace because it is early.

Even if training turns out to be easy, he says, adoption is a change-management and policy problem, and some people still lack an email address. His guess is that within two to three years the technology will be close enough to force policymakers and businesses to ask what to do with it.

On reports of failing enterprise pilots, he says the claims are partly true. Getting from 60 or 70% accuracy to something robust is like adding nines of uptime: each looks small but demands an order of magnitude more effort. He thinks the 95% failure figure is a bit hyperbolic, since spinning up a pilot is so easy and many are abandoned. Projects that get legal, policy and regulatory approval and reach an accuracy everyone trusts take six to 12 months, and the payoff is large when they work. His summary: easy to learn, hard to master.

Droege says he treats customer statements skeptically and looks at their underlying incentives, which are not always financial and can include ego or career growth. An executive sponsor for enterprise software, for example, needs to trust that you will make them look good. His shorthand: show me the incentive and I'll show you the outcome.

At Uber, his job was to find new businesses, and he knew nothing about restaurants. Owners were reluctant to share their unit economics, so the team ordered food, matched weights of ham, cheese, bread and lettuce against a supplier's catalog, and built an independent estimate of ingredient and labor costs. They triangulated it against what restaurateurs and industry insiders said. Roughly 20 to 30% of a meal goes to ingredients, 20 to 30% to labor and about 10% to real estate.

That gave them confidence to ask for 30% of the bill, which restaurants called far too high; the real clearing price was about 25%. The key insight was incremental demand: tripling orders with the same labor makes a 70 to 80% incremental margin, even if restaurateurs would not accept that framing. Droege says the biggest thing people miss is buyer urgency. Saving 10% on ingredients matters annually, but daily the owner asks whether people showed up and whether they made money, and a product that does not address that faces "a long road to a small town".

With so many smart people trying everything, Droege says a founder has to ask why they have an insight others lack and why they should be the one to act on it. The answer might be an unusual position, or a contrarian temperament that looks for true things others do not believe. If your research is shaped by what the world around you is saying, you will not have independent insight.

He adds that founders need a reason to work on a problem for five to 10 years, and that hearing a customer's problem is not enough. They also must not fall in love with their ideas; he describes part of the job as throwing away who you were and what you thought for the mission of helping a customer.

For the bar on new businesses, he says the most important factor is a founder who is a force of nature over a long period and can keep pivoting. The second is knowing which markets and models are good: marketplaces, SaaS and businesses with recurring revenue, stickiness or network effects, which are more valuable at scale than at low scale. Filtering ideas that way quickly eliminates the bad ones, after which you can choose the one you are passionate about.

Droege keeps a wide aperture on ideas for as long as he can. Before food delivery, Uber tried about 15 versions of other ideas. One was a van-based convenience store: 10 trucks with about 250 products in Washington, DC, which drew almost no orders, because it lacked cigarettes, beer and the other staples that bring people into a convenience store. Grocery's pick-and-pack economics scared him, and point-to-point delivery flopped because in 2014 consumers did not need it.

Food delivery worked on every signal, and it let independent restaurants compete with chains without needing prime real estate. It launched in Toronto in December 2015 and did about $20,000 in sales within two hours. About four and a half years later it reached around $20 billion, and he says it is now pushing $80 billion. He credits luck as part of the game too, and says COVID took it from about 20 to 50 in a year, shortly after he left.

On McDonald's, he first said no, since his vision was to help small restaurants rather than chains. McDonald's approached them anyway with its 80 million daily consumers, and he pushed them off for four or five months until his team called him insane. They ended up with an exclusive relationship and, he suspects, a better deal. Chains had avoided delivery networks because of basket-size economics, and his answer was to work it out by reducing the delivery radius or adjusting prices. Onboarding in about six months with two operations managers in New York was mayhem.

Droege treats gross margin as one coarse filter, not a perfect one; low-margin giants such as Costco and Walmart are great businesses. Still, high margins combined with healthy churn curves are a good sign. If someone pitches a 40% margin, he asks why it isn't 60%; the answer usually reveals a customer alternative, such as an offshoring firm running at 20%, which will pull margins down unless you differentiate.

The follow-up is why a competitor can't do the same. A low-margin strategy can work, as at Costco, if you use price to reach scale first and build habits, supplier relationships and operational know-how that others can't easily replicate.

He also believes in not losing as a precursor to winning. Tech culture, shaped heavily by investors, preaches "just go for it", but he says the best entrepreneurs examine the risk profile of decisions and try to make asymmetrically positive ones. Most founders fail by giving up before their timing, insight and product line up, so survival matters, and he thinks the current hype cycle makes this easy to forget.

His own cautionary tale is the used-golf-club business he started on eBay after the 2001 bubble burst. It made real money until he imagined buying every used club in America to become the market maker, and it stayed painful even though it was profitable. His life motto is "the end is never the end": when a decision feels impassable, remind yourself there will be another day.

Droege says some roles, perhaps 5% of a company such as AI researchers, need deep specific experience because the market moves too fast to train people up. For everyone else he interviews for three things: whether a candidate is a curious problem solver who can articulate it, whether they are humble enough to work across people, and whether they are a good leader. Because the world is changing, he values adaptability over one-to-one relevant experience.

At Uber Eats he tried to compose a management team as an organism of complementary strengths with minimal conflict. For the most part the same team ran the business from zero to $20 billion. He believed that people who know each other's strengths and weaknesses beat the classic objection that someone hasn't seen this scale, since he had learned scale himself.

In the AI corner, Droege says he uses AI as a tutor: coming from consumer products into a technical field, he talks to it in voice mode on the way to work to learn concepts that researchers lack time to explain. He also asks it for the most important thing in internal documents, then reads the document to double-check.