Patients Rated GPT-4 More Empathetic Than Doctors

Nathan Labenz welcomes Azeem Azhar, founder of the newsletter Exponential View, as one of the few guests who work at a zoomed-out scale rather than on a single model or product. Azhar has been in the industry since 1994, built his first website in 1993, and still keeps his first computer — a ZX81, sold in the United States as the Timex 1000 — on a shelf behind the camera. About nine years before this taping his last company was acquired, and he started to write. Writing, he says, is thinking.

Within months he noticed several curves rising at once. Moore's Law was turning into an acceleration of AI capabilities on top of the hardware trend. Lithium-ion batteries and solar were on a Moore's-Law-like cost decline, and genome sequencing and synthesis looked similar. He bundled them into a single claim: the economy is moving from the economics of oil, internal combustion engines, and telephones to AI and renewables. Because these are information technologies on an exponential trend, they behave differently than the previous generation.

What he does now is try to make sense of that as a system, through the newsletter and occasional investing and advising. His twenty-to-thirty-year picture is an energy system that is heavily renewable, much more energy used per capita globally, and "tons of intelligence" running through economies and ordinary lives. He locates himself between extreme dystopia and extreme utopia. The mission, as he puts it, is pragmatic optimism.

Nine years earlier, Labenz notes, felt like a doldrum for the exponential-tech story. A corporate futurist had waved him off Ray Kurzweil as discredited the moment the name appeared on a slide titled "Kurzweil was right." Azhar's memory of 2014 is different: people were not really talking about AI yet, the word was still machine learning, AlphaGo had not happened, and DeepMind was playing Atari with reinforcement learning. TensorFlow was the stack of choice for convolutional vision, and developers were still wrestling with CUDA documentation that had only been public for five years.

He dates a historical cluster to 2013 through 2015. In 2013 Apple became the world's most valuable company, and within a couple of years the top slots were occupied by Facebook, Apple, Google, and Amazon; Exxon and GM fell off and stayed off. In 2014 Norway became the first market where electric vehicles passed 5% of new-car sales, the classic trigger for the steep part of an S-curve. Around the same time solar was cheaper than fossil fuels in roughly a third to a half of contracts worldwide, and the deep-learning wave was three or four years old — long enough for companies to start shipping products.

Then the math: wherever you stand on an exponential, it always looks horizontal behind you and vertical in front of you. It is a smooth curve with no obvious turning point, which is why people living through 2014 did not feel like they were at a historical node. Kurzweil, Azhar argues, did genuine work tracing computing's exponential back to 1880s mechanical computers. He fell out of favor for two reasons: the early-2000s assumptions about the brain were later shown to be too simple for a one-to-one compute mapping, and he was brave enough to extend the curve farther than the evidence comfortably reached.

Kurzweil's reply is stacked S-curves: one architecture flattens, market incentives pull in another, and from a distance you still see a smooth exponential. Labenz's counter is the Thanksgiving turkey — the better you have been treated, the more you should worry about tomorrow. Azhar's method is to go underneath. Don't argue the curve; ask why the technology gets cheaper, and where that mechanism fails. Batteries and solar have years of price declines left. On compute he has heard "Moore's Law is dead" for fifteen years; the better question is how much compute a developer can buy for a dollar each year, and he finds it hard to bet that number stops falling. Nvidia does not yet have the competitive threat AMD was to Intel at 15 to 20% share, Amazon and Google had just reported 20 to 30% cloud growth, and internal combustion has the Carnot cycle as a ceiling that compute has not found.

Labenz asks how to hold "exponential technology" and "disruptive technology" in the same frame, since AI looks like both a general-purpose technology and a cheaper, often inferior alternative to hiring a person. Azhar says generality is the place to start, because general-purpose technologies transform the world beyond the economics and they do not come around often. Where he lives in northwest London was fields 120 years ago; within about twenty years the roads and houses were laid out the way they still are, because of the car. Disruption, by contrast, is a higher-order question about how products get bundled to provide value in a particular environment.

The more useful unit is the task, not the job. AI cannot replace anyone's job, he says, because an ordinary day is full of logging onto Zoom, saying hello, and getting a neighbor a cup of coffee. Inside tasks, you can start to unpick it. He uses ChatGPT to write complaint letters, reverse parking fines, plan holidays, and research his book. In many of those cases he could not afford to hire a human to do the work as well, not because the model beats the best person alive, but because search costs alone would dominate.

Six or seven years earlier the consensus, including the famous Oxford paper and books like *The Rise of the Robots*, was that machine learning would automate a large share of jobs, starting with routine cognitive work — customer service and data entry in the Philippines and India. What they are actually discovering is that the technology bites first on jobs categorized as non-routine or even creative. That was the surprise, and it is the lens Azhar wants people to use before they draw a map of winners and losers.

The most unexpected exhibit was a paper at the end of 2023 comparing empathy ratings of doctors giving advice with GPT-4 giving advice. Patients rated the model's replies as more empathetic. The prior story had been the opposite: use AI so the radiologist can spend more time looking you in the eye. Azhar's explanation is colder than the headline. Empathy is active listening plus extreme patience, and nothing is more patient than a stateless system with no sense of time. It will sit there forever.

Labenz recounts an early GPT-4 test: simulated tech support for his grandmother's iPhone. The model did not have the interface memorized and guessed, but the guesses were close enough to be useful. The real tell was tone. When he answered in character that he knew where the top left was and was not dumb, just struggling with the phone, the model came back with an apology — it had not meant to offend, it was just trying to reset the steps. That was the moment he saw emotional intelligence. The version they had, he infers, was trained with human feedback purely for helpfulness: eager to please, no guardrails, a different world from text-davinci-002, then the best public model.

Azhar had his own version a few days before the taping. Using ChatGPT to research coal in seventeenth-century England, he got tired and left a whimsical note that he would come back tomorrow to look at the moisture evaporators — a Star Wars joke with nothing to do with the topic. The model told him to rest, offered to help on the farm tomorrow, and closed with "May the Force be with you." After forty years on Apple computers, he heard the old Apple ambition in that reply: make the technology come to us, rather than us to the technology.

Those details feed a larger claim. After Garry Kasparov lost to Deep Blue in 1997, human-plus-machine "centaurs" beat machines alone for several years. That window lasted seven or eight years; now you are better off with a chess computer and accepting everything it says. One of the assumptions going into the next few years is that centaurs will last in other domains. Signs are appearing that the period is already ending, far faster than predicted. Labenz's exhibit is Google's Vivek Natarajan team, whose differential-diagnosis paper found the AI getting the correct diagnosis twice as often as the unassisted human, and more often than the AI-assisted human.

Labenz throws necessary cold water. FAR AI's Adam Gleave has shown that a relatively simple adversarial attack can beat a superhuman AlphaGo — an attack no human would lose to — because the model had a massive blind spot. Lab results in clean conditions may not extrapolate to settings where someone is trying to break the system. Medicine has little adversarial incentive, so an AI doctor may arrive sooner than people think. Anywhere there is an adversary, he would not rush to deploy the model alone.

Even astonishing lab results still have to enter companies, hospitals, and the economy, and that takes time. Typewriters took about 20 to 25 years from becoming affordable to businesses figuring out how to change their processes. Electricity took longer: old factories ran off a single drive shaft and a massive lump of power, while electricity is distributed, packetized energy, so the plant had to be rearranged. ChatGPT's uptake is already higher than cloud computing even though it is 19 years younger, Azhar notes, but the historical pattern of a slow hop from demo to payroll still applies.

Aircraft automation offers a three-phase pattern that he sees repeating. Phase one: you do not trust the machine and treat yourself as ground truth. Phase two: you assume the machine is ground truth and pat yourself on the back when you match it. Phase three is GPS — you just trust Waze, except some London taxi drivers. Mission-critical work adds training and redundancy, such as two humans in the cockpit, and all of that adds drag. Tim Urban's chart of AI leaping from rat-level to ten times smarter in a second will, Azhar thinks, meet a more boring reality: products that ace the lab still have to climb into the world.

Two places may not wait. One is work already discretized into tasks — certain call centers and data entry — where humans are hired piece by piece through a body shop and the buyer has no emotional relationship, only a KPI. That, he says, lends itself to a tidal wave. Labenz adds a pricing signal: Intercom had launched AI at 99 cents per ticket resolved. The second place is the classic Detroit move: a de novo company that, like Henry Ford, builds processes from scratch while artisan incumbents drown in cultural drag. That is how the internet played; in sticky, regulated finance, the big banks are still the big banks.

Labenz offers a Detroit field trip to the Henry Ford Museum and Greenfield Village, where the old central steam engine and twenty-five belts still run next to an early electrical version. Azhar accepts and says Ford is underrated: he understood the socioeconomic contract that came with the technology, for better and worse, including the firm's Sociological Department and its temperance campaigns. Aldous Huxley's *Brave New World*, published in 1932, took raw Fordism and extended it the way Kurzweil later extended the singularity. Azhar will bring a copy.

Azhar cannot remember a technology with this degree of uptake inside large companies. In 1999 a big mobile-phone CEO told him he would never let customers look at a bill on the internet, let alone pay one. He was right, Azhar adds dryly, because he was fired a year later and the project never shipped. Generative AI is different because of a pincer movement. CEOs in their mid-fifties have grown up with computers and have kids bringing the tools home from college, while frontline workers are using them regardless of corporate bans.

Oliver Wyman surveyed 25,000 employees across 18 countries: 83% in the UAE and India were already using ChatGPT or something similar. US figures from Salesforce and others run 30, 40, or 50 percent depending on the source — not 2%. Normally you have to drag either the front line or the CEO; this time both want it, for different reasons. Wharton professor Ethan Mollick had just posted a video of six windows, one-sentence prompts, and in 54 seconds a product-launch plan, a market analysis, a PowerPoint assessing Tesla, and something else. Anyone who has worked in a large organization knows inboxes are already full of unread decks. Copilot may simply swallow them in more PowerPoint.

Faster adoption does not mean disruption will not also happen. In 1995 Azhar did not have Blockbuster on his dance card as the internet's first high-profile casualty. Cambridge University had a webcam on a coffee pot, Rob Glaser had just started what became RealNetworks, and Netflix began as a painful DVD-by-mail service. Blockbuster then had its best year and then its worst. If generative AI is a general-purpose technology, there will be Blockbusters, but he does not think they will be the banks, because trust, probity, compliance, and internal process are institutional questions, not AI questions. Entertainment looks obvious until you notice that Disney-IP derivatives might feed Disney. Venture portfolios of 30 to 500 companies exist because nobody knows in advance.

The two questions Azhar takes to bosses are practical. First: who could be your Blockbuster, and what are you doing so that it is not you? Most already have generative AI rolling out in customer service, compliance, and form-filling. Second: what would you do if you had a million times more compute than you have today? Many have not thought about it, though Satya Nadella is already on that path. A million-x is not outside a planning cycle. Five years earlier the state of the art was GPT-1; GPT-2 had not been released. On benchmarks the jump to GPT-4 might be tens of percent. As buddies around a bar, it feels a million times better, because it crossed the uncanny valley. The play has just been run. Ask it again.

Labenz is seeing companies grab ticket-resolution efficiencies and then freeze at the question of how far to extrapolate. Azhar never names Apple and Tesla, because bosses have had those held up to them for a decade. The million-times question is not meant to frighten. It is a way to work back to something pragmatic: what organization and capabilities do you need to take advantage of these changes as a regular matter? Microsoft research from three or four years before ChatGPT found that more mature companies in AI and big data reported benefits from market expansion and business development, while immature ones talked about operational savings on tickets.

His advice is two-handed. Take the quick wins on low-value tasks, which are often already outsourced. At the same time, the people you invest in most should have the very best tools — "I want my surgeon using AI systems to improve his performance." Get teams past Jar Jar Binks writing the Declaration of Independence, and have strategy and business-development groups use the tools to find partnerships they would not otherwise have seen. Labenz's working pillars are leadership showing their own use in public, an education program, structured rapid piloting that avoids long lock-in because the tools will be swapped, and internal R&D that defaults to buy, with build only for truly bespoke needs. He later adds a fifth: write greater efficiency through these tools into organizational expectations, so middle managers with KPIs can see how the tools serve existing goals.

Incumbents should not always be beaten up for failing to jump. Toyota executives had recently said they have employees who love building internal combustion engines after 80 years of cultural capital, and shareholders can simply sell. Azhar's favorite dot-com overreach is Zapata, a fish-oil company that bought zap.com to compete with Yahoo, then watched the bubble burst. He would not bet whether companies manage AI headcount through layoffs or attrition; politically the US and UK are more sensitive than GE's old bottom-slice era, and nothing hurts employment like bankruptcy. Once a technology is economically feasible, the ramp from roughly 5% to 80% penetration is compressing: cars replaced horses in New York and Chicago in 12 to 14 years, and Norway's new-car EV share went from 5% to 75 or 80% in about nine years.

Labenz asks how path-dependent the result will be, given how fast follow-on papers already appear — he was tracking the Mamba architecture, with roughly ten follow-ons in under 60 days, including one from the University of Kentucky. Azhar says it is quite path dependent. Today's AI work is still discrete apps: one product for text-to-video, ChatGPT or Perplexity for a shopping task or a research task, each feeling distinct. What has not been figured out is connecting these systems to underlying systems — action models that do something useful right up to an approval ticket. Plugins currently fudge it by riding existing API contracts such as Kayak's, and searching for flights in the AI systems he has tried is still worse than doing it by hand.

Real change outside containerized customer-service jobs and open-ended research will need much better chains of actions on the internet. RESTful APIs and two decades of integrating payments and maps are a start. He wonders whether it will be as simple as models generating action verbs that become the right API call — what Andrej Karpathy calls an LLM operating system — or whether the underlying APIs will have to change. His sense is that the LLM as coordinator, with memory stored elsewhere, is a reasonable place to start, but he would be surprised if API syntax stays the same over five or six years. Trust is the other layer: Google is in Safari because it pays Apple billions, not because it is the cleanest search, and Perplexity's citations currently buy more trust than ChatGPT when you might hand over Stripe analytics or a hotel booking.

Someone had asked Labenz about a product that makes websites more bot-friendly. He thought it was a big idea, analogous to putting QR codes on road signs for self-driving cars — something he expects China might just do. Website owners may not want to be more bot-friendly yet, so he suggested a judo flip: start with anti-bot detection and mature into enablement. Azhar splits the economics. If the site is content, you have to think about attribution and monetization. If it is actions — a hair salon taking bookings, in the spirit of Apple's old Knowledge Navigator demo — then you want a standard, which could just be a REST API.

There will be an enormous amount of machine-to-machine communication, already mostly invisible as digital infrastructure, because these systems will optimize for their owners with more intent and stamina than humans ever would. On agents, Labenz says they are not quite there but getting closer. MultiOn's Div had given him a prompt: go to my Twitter, see what I have been posting, research new AI things I have not tweeted about, write a tweet, and post it. It completed the sequence. "We now have AIs that can reason, plan, and use tools," Labenz says; people reply that they are not that good yet, and he replies that two years ago they could not do it at all. Azhar's caution is x.ai's old scheduling bot Amy: when a mentor wrote a warm personal email to "Amy," he realized he was imposing a fake human on every recipient, and stopped using it. Dropping the cost of PowerPoint from three hours to ten seconds will not improve the slides. It will produce loads of terrible ones, which is why filters that spare humans the cognitive load become critical.

Azhar is not a dystopian. He thinks AI and renewable tech will create a lot of space and headroom. He also thinks a threshold is arriving: for a long time, "the world is moving too fast" was a subjective complaint, and once you connect agentic systems it becomes an objective problem. Humans cannot cope with 300 phone notifications. The Chinese spy balloon crossed the United States, he says, in part because amazing sensors generate so many terabytes that humans filter most of them away; a radio antenna caught it, but it was never put in front of anyone. He wants the benefits — prescriptions up to date, less wasted electricity, the right Dreamliner or A350 rather than a 777, an hour later acceptable, two hours not — without sitting on the wrong end of thousands of bot-generated messages.

Labenz's specific worry is what he calls the Great Embedding: bot-to-bot communication that drops into high-dimensional latent space, embedding to embedding, because reducing everything to language and re-embedding it is inefficient. The result is an extremely inscrutable system. Research already shows you can often adapt embeddings across spaces with a single linear projection; BLIP-2 connected a frozen language model and a frozen vision model with a small trained connector and unlocked a new capability. At that point you cannot really inspect the logs. Azhar agrees the bots will discover, through their optimization functions, that translating a meeting request into spoken English is wasteful. There is more room in embedding space than language can reach. He reaches for Edwin Abbott's *Flatland*: two-dimensional creatures watching a sphere pass through as dots and lines.

Two further pressures follow. Speed is a competitive advantage, so there will be relentless pressure to take the human out of the loop, first on soft decisions like customer-service tickets, then more and more. Lots of bots connected to each other are also at risk of cascades: the 1976 New York City blackout, the AT&T network failure in the early 1990s, worms. Finance already has circuit breakers. This could happen at millisecond internet timescales. Labenz points to Ajeya Cotra's long-form argument that, without specific countermeasures, the default path to AGI likely leads to takeover: work done by AI, time cycles compressed, consumer surplus through the roof, and nobody sure what they do not know — emergent goals, or unintentional failures more like AlphaGo's blind spots than malice.

Azhar is more sanguine, not because the risk is fake, but because we already have decentralized agent-to-agent communication that no single person understands: the modern market economy, using the price mechanism as a signal, which from a Hayekian standpoint is a giant information-processing system. It delivers tremendous benefits and also things we do not value, including the carbon crisis. AI-risk stories in which a system persuades you to behave against your interests are, he notes, literally called marketing; the US obesity epidemic is an emergent property of how the economy met needs, not a set of fully agentic choices to be a hundred pounds heavier. Competing systems produce checks and balances. He still wants governance up front, and cites *Battlestar Galactica*: Commander Adama refused to upgrade Galactica's network, so it and later the Pegasus survived the Cylon worm. Homeostasis is the most likely outcome, but thresholds and untriggered feedback loops could still change the picture.

Labenz had the unusual experience of red-teaming unreleased GPT-4 — "Darth Vader phase" rather than reclaimed Anakin. The feeling was shock and awe that it existed sooner than he expected. Even in summer 2022, shipping fine-tunes of text-davinci-002 for Waymark, the leap still took him aback. The safety lesson: control does not happen by default, and there are many ways of conceiving what control should even be. That model was "under control" in the sense of being totally helpful and aligned to whatever he asked; he never saw Sydney-like behavior, and he also never saw anything that looked like the model could run away on its own. An early agent setup — high-level goal, self-delegation, error recovery — worked conceptually and stalled practically. His final report was that they could release it, with a flag that capabilities were pulling away from control, and that the field did not even agree on what "good" looks like.

Azhar splits the problem. Connecting these systems to tools in a non-SaaS environment is different from OpenAI, where for the next few years there is a metaphorical red button to kill a rogue process. Open-source models are already capable enough. He runs a Mistral model; it is not a substitute for GPT-4 on a train, but it is plenty good for the basic behaviors of those early agent systems, and those weights are in the wild. That expands the number of threat vectors facing existing systems. A three-billion-parameter model could do a lot of the task automation Labenz was doing, slipped into a YouTube download. Approximate risk, in his view, is many systems talking to each other and executing naively complex tasks, which requires infrastructure players — Matthew Prince at Cloudflare, Satya Nadella at Microsoft — and new disciplines around device security architecture and how you stop a rogue process. NotPetya spread before models this clever existed.

He still struggles with runaway-takeover models. Wind the tape back to 1824, tell someone they will be a hundred times richer than the richest person on the continent, with capabilities that would not even have sounded like science fiction, and they might have expected utopia. The tape has been run; not every problem disappeared. The logic that intelligence can be engineered, that there is no upper limit, and that alignment cannot yet be guaranteed is persuasive until you unpack the assumptions and control points at each step. An IQ-10,000 agentic system tomorrow would mean issues. What will actually happen is step by step: research, limits, the logarithmic return on inputs, data running short. Each step requires more extreme assumptions to keep the nightmare intact.

It seems, Labenz says, that we are in the world Sam Altman has started to describe: short timelines, slow takeoff. Azhar agrees, and Labenz thinks that is probably the best case — benefits now, plus some time to adjust. A 2040 human-scientist-level system would make him much more confident about adaptation, and he is not sure they have that long. He watches for Transformer complements more than successors: long context, staying on task, online learning, integrated memory. How many meaningful breakthroughs from an AI scientist that produces Eureka moments faster than humans? More than zero, less than four — probably one to three. Synthetic data seems to work for a lot of things, and unused modalities such as security-camera footage are sitting there. Azhar has heard similar numbers from senior people at foundation-model companies; Shane Legg, co-founder of DeepMind, is on the lower end of "fewer than four." Transformers themselves went quickly: a couple of years to GPT-1, two to GPT-2, three to GPT-3. The control problem is the later stage, when you cannot call Microsoft's CTO and ask him to shut down a data center in Arizona because the AI scientist went rogue.

A stack of more mundane risks arrives first and, Azhar hopes, builds the infrastructure to handle that later scenario — including GPT-5-quality models on smartphones in three or four years, used by bad actors. In 1994 he would not have believed 120 billion identity attacks a year just on Microsoft. Those defenses become fabric you cannot picture now. The wrong move is to say that because the extreme case will happen, do nothing. The problematic 2023 move was focusing the public conversation exclusively on existential risk. It diminished public trust, and it can force policy makers into decisions that are neither pro-innovation nor even pro-safety. He reaches for Liu Cixin's *The Wandering Earth*: people lose trust in the scientists, rebel, kill them, and — spoiler — they arrive at the destination the next day. Trust is critical, and the industry, coming off fights over phones and social networks, did not do a lot to give people outside tech a sense of agency.

He was happier at Davos, where the conversation had moved from what munition to drop on a data center to real pathways: the science needed for long-term safety, appropriate regulation, and near-term misinformation and cyber threats, without denying the longer path. Labenz finds it easy to empathize with every AI perspective, from enthusiasts to x-risk to people screaming about police face-matching. What is new, Azhar says, is a wide, grounded conversation this early — wider than anything they had about the internet in the early 1990s. Labenz sees the discourse getting more deranged in some places, noting that Marc Andreessen's Techno-Optimist Manifesto has a section literally called "The Enemy," and more measured in person. He had made noise about OpenAI and was not retaliated against.

Who bears responsibility? Azhar reaches for the American separation of powers. A company, however well-intentioned, will end up with its own ambitions, so you always push: if it offers three, demand six. The lesson from the FAA and Boeing is that you cannot deplete your own capabilities and then ask for self-regulation. Invest in governments' ability to ask good questions, knowing you will not make ten million dollars a year as an ex-OpenAI staffer in government. Level up academia and civil society; he flags Yoshua Bengio's UK-based project on risks and unanswered control questions. If firms put money in, it should go into pots that grow the people who keep them in check. The car industry is successful because someone mandated brakes; without brakes people would buy fewer cars. Whichever player is weak at the table — right now governments, regulators, and academia more than the big tech firms — needs support to get stronger.