Infinite AI Interns Are Already on the Job

Nathan Labenz has spent the last couple of years living inside applied AI, as host of the Cognitive Revolution podcast, an AI investor, and chief AI officer at the video company Waymark. The conclusion he brought to Texas State University's computer-science club is blunt. Most institutions are struggling to make sense of what is happening, let alone keep up. Colleges, in particular, are at risk of failing students by continuing to lean on skills that, while still fundamentals in some sense, are increasingly best provided by AI systems.

He is not treating that as a thought experiment. He is hearing more software companies say that AI tools make them less inclined to hire junior developers, because the same tools create more leverage for senior people and promise better return on investment. The talk's title is the warning and the offer at once: ten practical tips for current students, or how to stand out in a world of infinite AI interns. He will not tell anyone to drop out. He will tell them to spend real time incorporating AI into their work, because this mega-trend will shape entire careers, and early-career people need to adapt as fast as possible.

The first job, before any tool, is calibration. He speaks to many audiences and is still amazed how many smart, accomplished leaders have remarkably little curiosity about how far the technology has already come. Everyone in the room has used ChatGPT. That is not the same as knowing the state of the art. His high-level description, the one he gives almost everyone, is that AI is closing in on human expert performance on routine tasks. The word *routine* is doing a lot of work. The systems are getting really good at things where we already know what good looks like.

On MMLU, a battery of college and graduate exams across disciplines, a typical human might get 35 percent right. A domain expert gets about 90. GPT-4 gets 86. Coding contests look similar: drop a model into the field and it will not take first, but it will sit above average. The sharper exhibit is Google's Med-PaLM series on medical question answering. Passing a U.S. medical licensing exam is 60 percent, a number Labenz says should give you pause about how accurate your human doctor is on any given topic. Med-PaLM 1 cleared 67. Med-PaLM 2 blew past the pass line and reached 86, again closing in on expert level.

Researchers then asked human doctors the same questions, and asked other doctors to judge whose answers were better. On the desirable traits — better reflection of medical consensus, better reading comprehension, and the rest — the AI took the higher marks overwhelmingly. The only category it lost was hallucinations, described in the paper as more inaccurate or irrelevant information. On the other things you do not want in medical Q&A, the human doctors did more of them. As judged by human doctors, the model beat human doctors on eight of nine evaluated criteria. Versions of the same pattern show up in diagnosis. These results, he says, are happening across the board.

What the systems still rarely do is come up with genuinely new, high-quality ideas. He calls those Eureka moments. He used to say there were none; now he says there are precious few. One paper is, uncannily, titled Eureka. The task is writing a reward function so a robot can learn to twirl a pen. At the start the robot is flailing, the reward is sparse, and there is nothing to learn from — the same trap that led OpenAI to abandon a web-browsing RL project a couple of years ago. Human roboticists hand-write proxy rewards, angular momentum around an axis and so on, to catch the flicker of progress. GPT-4 writes better reward functions than those specialists, and the robots train better. Even so, you still see very few cases where the system outperforms genuine experts at novel tasks. When it happens, it makes the news, because it is still rare.

To explain the pattern, Labenz uses a homemade scorecard he calls the tale of the cognitive tape: break cognition into dimensions, then score a human expert against an AI, not an average person. Breadth is already a blowout. The models have read the internet and the books, can sit every exam in every subject, and speak every language. An individual human is super narrow; humanity as a whole is closer to the machine. Depth still belongs to the expert. The systems are getting good, but they do not yet have a specialist's command of a subject. Breakthrough insight is the same Eureka gap, and he gives the AI a one. That, he says, is humanity's biggest edge right now.

Speed and cost run the other way. Models typically generate faster than we can read, a notable threshold, and you should expect at least ten times faster and probably at least ten times cheaper. They are available and parallelizable: on 24/7, idle at no cost, and you can spin up ten or a hundred at once. Memory, fallible as ours is, is still much better. We carry an integrated sense of who we are, what we are doing, and how this hour fits the long game. Today's systems have factual memory of a sort, but querying on the fly is brittle, working memory is finite, and retrieval-augmented generation is one patch among several, very much a work in progress. Bedside manner is the contrarian row. Doctors are overworked; chatbots will explain something ten times and stay polite. He gives AI the higher mark, at least in some ways.

Two more rows matter for careers. Technology diffusion is slow in people, especially mid- and late-career workers who will be reluctant to change how they work. "You can't teach an old dog new tricks" is, in his view, a lot of the opportunity for young people walking in. AI itself diffuses the other way: a 40 percent training speedup gets plugged in and everyone has it. That pace may be slowing a little as top labs share less, but it is still fast. Adversarial robustness is, alongside Eureka, the other giant weakness. Humans get a bad feeling about a scam or a person on the street. Models do not. Jailbreaks, leaked information, car-dealer chatbots agreeing to sell a car for a dollar: they are easy to trick. This is not only a chatbot problem. FAR.AI found an adversarial attack that consistently beats superhuman Go systems with a strategy a human would never lose to. Superhuman players still have huge blind spots. Keep the scorecard in mind, he says. Play to human strengths. Do not compete where the systems are already superhuman.

The mid-career version of the advice, including from Labenz himself, is to think of an AI as a day-one employee: pretty bright, eager, hardworking, totally lacking business context, and capable of weird mistakes. You have to spell everything out, give clear directions, and show it what good looks like. If you do that, you have infinite interns, or in a CS setting, infinite entry-level software developers. No software firm in human history could say that. It is starting to be a thing. At a recent event he heard companies say they are not hiring as many junior coders, because they would rather make seniors more productive with tools and infinite interns. He will not sugarcoat it. People entering the workforce are now competing against that stack, and they need to angle toward human strengths.

Working with the stack means knowing which mode you are in. Copilot mode is the ChatGPT experience: you are doing the work, you ask for help, you look at the result in real time. You are the pilot; it is the copilot. You must evaluate every output. Lawyers have pasted ChatGPT hallucinations into court filings and lost their licenses. That is malpractice. Delegation mode, or task automation, is different. You find a bottleneck, lock down inputs, outputs, and triggers, test until you can trust it, and then you stop reviewing every item. The trade-off is a narrower scope in exchange for a process you can actually run. Agent mode is the dream in the middle: delegate in real time the way you do with a copilot, and trust the result the way you can after a delegation setup. "Find twenty Detroit accounting shops, pull their rates into a spreadsheet, write an analysis, come back when it is done." That is not quite there, he said in the spring of 2024, but the field's consensus was that the next big OpenAI release would power a lot of those multi-step, multi-app cases. Having the vocabulary, and being able to explain it to other people, is itself useful.

Use the best available systems. Free ChatGPT is not good enough. Twenty dollars a month is worth an Upwork gig if that is what it takes. When he wrote the slides, Claude 3 had just taken the top of the leaderboard; by the time he spoke, a new GPT-4 version was probably back on top. The churn is the point. His own stack was Claude 3, GPT-4, and Gemini 1.5. Open-source options are decent compared with a year earlier, but they are hobbyist land: the best ones are large, and you have to figure out where to run them. His advice to business owners is not to cheap out. His advice to students is not to work at a place that will not buy the best tools.

Copilot craft is prompting, and the official guides from OpenAI, Anthropic, and Labenz himself mostly say the same things. Instructions should be clear, accurate, and unambiguous — an art, and a useful one, because it forces you to think before you start typing `class`. Examples often beat long descriptions. The GPT-3 paper was titled "Language Models are Few-Shot Learners" for a reason: a couple of input-output pairs can teach a task you never described. We are only a couple of years into that era. Roles help. "You're the senior engineer doing a code review" is one version. His personal mashup is Einstein-Hemingway: very smart, and terse. He will also ask the model to demonstrate intelligence through economy of words. Labels, chain of thought, and an explicit output template all still work. Your future boss will probably not be good at this. A short crash course is a huge return in time and energy, not just money.

Anthropic's meta-prompt is the next twist: paste a rough prompt into their notebook and it writes you a better one. The systems are already helping us prompt. For code, GitHub Copilot is the common one. Codium focuses on integrity — unit tests, type checks, the homework dings and code-review dings Labenz was never great at. Cursor is the AI-first environment. He asked who in the room had used it. No hands. "If you get nothing else out of this you should all go try Cursor." Watch for hallucinations. The models make fewer mistakes than they used to, and will make fewer still, but they still make them.

Delegation mode is how a junior can blow a boss's mind and walk into rooms they would not otherwise enter. Most of it can be done on no-code platforms. The fact students need to understand about real organizations is that processes are often undocumented and barely formal. Work gets done, but nobody sat down and designed a flowchart. He titles that slide like the old GAN demo: these process diagrams do not exist. People know their own handoff. They do not necessarily command the bigger picture. Success starts by mapping the territory: inputs, outputs, what happens in between, whose job it is, what starts the process, and what happens at the end.

His other rule is prompt before app. Get ten real inputs and ten good outputs from the person who owns the process, and prove the model can do the core task before you touch Zapier. A lazy version often works: feed it nine examples and ask for the tenth. If the core works, you can pipe, trigger, and maybe write a little custom code inside a Zap. The jobs are not sexy. Customer-service tickets nobody enjoys, a first draft from a hundred past replies, sometimes still a human in the loop. In a room where nobody else knows how to do that, you are immediately a difference-maker. The experts already in the building have something you do not. They do not have task automation. They will love you for bringing it, not least because it is faster and cheaper than another headcount.

A student put the gap on the table. Class time is stacks and queues. ChatGPT already understands that material. None of the talk is in the curriculum. Should they do the bare minimum for school? Labenz will not romanticize the catalog: school is a little messed up in a lot of cases. In the work-for-pay world, almost nobody cares whether you used ChatGPT. They want the job done well and fast. Data-security holdouts exist; he thinks many of those concerns are overblown, and there are ways to protect privacy. He is not telling anyone to drop out. If you stay, pass the closed-book exams, retro as they are. He barely codes by hand anymore. He describes what he wants, pastes in a class or some docs, and lets a better typist generate it. The old line was good, fast, cheap: pick two. AI is breaking that, or at least redefining the triangle. He routinely delivers all three. People love that, and almost no one calls it cheating. He would not work for a boss who did, unless there was a serious reason, like a military contract.

The easiest install he can name would still change how a lot of organizations work. Claude for Sheets wraps an API call in a spreadsheet function: structure data, fill gaps, answer questions, with a caching layer that makes the cheap tier extremely cheap. He estimates 0.1 percent of businesses have it. Walk in and ask whether the company has tried Claude for Sheets. Nobody will know. That is immediate alpha. He describes himself as an AI scout: all his time is trying new things and reading research. What he finds remarkable is how often people cannot be bothered. Zero Cursor users in a computer-science club was a warning sign. Gamma makes slides. Descript edits video. Suno and Udio now make music he would actually listen to for fun, lyrics included. Youthful energy spent trying the new app is an edge mid-career people will not take, and it routinely blows minds.

He had, that day, given Julius a YouTube URL and asked for an MP3. It wrote the code, ran the code, and handed him a download link, first try, in about fifteen seconds. ChatGPT might refuse, or write code you still have to run somewhere. Claude might write the script and leave you looking for an environment. A browser that just does it is the product. Devin, the coding agent a few students had heard of, was still waitlisted for him, getting good but not quite. The important behavior is the loop: hit a bug, go read the docs, come back, try again, change the plan. Open Devin you can run yourself. Magic.dev was another name coming, allegedly a huge deal, still opaque. The agent era, in his telling, is the intern who does not clock out after the first failure.

Once a delegated system is running, a new question appears: is this working, and how would we know if it quietly got worse? Evals are ongoing unit tests for AI inputs and outputs. Public versions are called benchmarks; internal ones are evals. At Waymark, models write video scripts for local restaurants and law firms, and contact details have to sit in the template's right slot or the video looks wrong. Checking a hundred scripts by hand is tedious, expensive, and inaccurate as attention drifts — perfect AI work. Labenz used Claude for Sheets and the meta-prompt to build a checker: URL in the wrong place is a violation, missing phone number is a violation, a clean script returns true. Humans write the rules; the model does the grinding. Five good examples and five bad ones are enough to start. Someone who can build that, he says, is a unicorn in 99 percent of businesses in the country today. Mid- and late-career colleagues will have expertise you do not. You want to be the person who can lead them through what they do not know.

The new job titles he sees are still in flux. The AI engineer is a software engineer who can actually put models into products. The ML ops specialist fine-tunes, curates datasets, and runs evals. The AI implementation specialist is less technical: an internal chatbot over the knowledge base. He stood one up for a thousand-person company with Chatbase and 250 documents; half the questions that used to hit a human team moved over. Mantras help people who are cold to the technology. Inside Waymark he says "AI or die" and "done for you beats do it yourself." Three general ones: summary, not strategy — business strategy is still on you, the Eureka gap again. Process, not product — do not delegate the thing that makes you special. Convert, not create. And keep updating the world model. GPT-4 was the moment a naive user found the tools often useful, and that was only a year old. The cognitive tape will keep changing. Breadth is so vast it takes a community. He trusts a short list: McKay Wrigley on Twitter for coding demos, swyx and the Latent Space podcast, and Wharton professor Ethan Mollick for general knowledge work, the one business-school name a future boss might already know. Do not buy the course that says 99 percent of people use ChatGPT wrong. Real information is free. Buy GPT-4, not prompting secrets.

A finance question gets an honest hedge. He does not know the sector well, because it keeps methods as trade secrets; swyx had just said the same. High-frequency trading has been an AI game for years, usually on narrow systems, not today's general models. Bloomberg trained on its proprietary data and still found GPT-4 better. Mortgages and document checklists, though, look like every other business: task automation on whether the forms match. On training your own model, you are almost surely limited to fine-tuning. From-scratch general models take huge data. OpenAI's API can start around a hundred examples for one task, a few hundred for decent performance, thousands if the task is hard. Quality is the whole game, which is why evals matter. Fine-tuning also narrows the system. Waymark's GPT-3.5 script writer, trained on hundreds of examples, can only write scripts. Ask it for a birthday-cake recipe and it literally cannot. PEFT methods such as LoRA and QLoRA let you tune a small slice of a Llama in a Colab notebook. He tried to fine-tune a model on his own essay voice and found it very hard. Dropping thirty essays into Claude 3 and saying "write in this style" worked much better. Software engineers, he warns, reach for LoRA when more examples would win. Go become the AI experts those dinosaurs will never become, and they will love you for it. His next audience was millionaire business owners asking the same questions. If you can answer them, they will hire you.