Ethan Mollick on Building AI-First Organizations

The Wharton professor explains how leaders, employees and experimentation can reshape work while preserving expertise and human agency.

EMEthan MollickInvitado · University of Pennsylvania
Canal
Sana
Publicado
5 jun 2025
Vídeo
1:00:20
Lectura
9 min
Original
inglés
Abrir en YouTube
Al desplazarte, el vídeo se reduce a la esquina inferior derecha y sigue reproduciéndose. Los subtítulos siguen tu idioma de lectura.
Idioma de lecturaDisponible en 2 idiomas
Este artículo aún no está en español, así que se muestra el original en inglés.
RESUMEN

En 30 segundos

Ethan Mollick joins Sana founder Joel Hellermark to discuss evaluating AI, redesigning organizations, incentives and the future of apprenticeship. He argues for ambitious experimentation and growth rather than treating AI solely as a cost-cutting tool.

  1. 1Build your own evaluationsTest real business tasks and have experienced people judge quality and errors.
  2. 2Choose growth over a narrow cost lensMollick warns that employees may hide gains if AI is used mainly to cut jobs.
  3. 3Leadership, lab and crowdA coordinated adoption strategy combines executive direction, broad experimentation and a lab that tests and scales discoveries.
  4. 4Protect the apprenticeship pipelineIf AI takes junior work, organizations need deliberate practice and coaching to build future experts.
  5. 5AI as a team collaboratorIn a P&G study, individuals with AI performed similarly to teams, while teams with AI more often produced standout ideas.
  6. 6Beware premature KPIsCounting outputs and cost savings too early can constrain experimentation and reward the wrong outcomes.

Conclusiones

  1. Evaluate AI against real business tasks, not only standard technical benchmarks.
  2. Connect leadership direction, a lab and broad employee experimentation.
  3. Fear of layoffs can keep employees from sharing useful discoveries.
  4. Protect the route from junior work to expertise through deliberate practice and coaching.
  5. Use AI to explore ambitious possibilities while retaining human judgment.
  6. Avoid narrow early KPIs that reward output volume or cost cuts at the expense of learning.
TEXTO COMPLETO

Lee el texto editado por capítulos

Solo la transcripción
01

AI history and useful tests

Mollick recalls early AI research and argues for practical evaluations of business tasks.

Ethan Mollick warns that an organization facing a new technology can choose to shrink costs or use its capacity to grow. He worries too many firms will choose the smaller ambition.

Mollick recalls helping AI researchers communicate their work while an MBA student at MIT during an AI winter. Projects then ranged from observing babies to Marvin Minsky's Society of Mind; he finds it ironic that large language models emerged from training on vast amounts of language.

Joel Hellermark asks which early ideas remain relevant, including augmenting versus replacing intelligence. Mollick says those once-hypothetical questions have become practical: powerful but limited systems force organizations to decide where humans fit.

Mollick finds AGI poorly defined and older intelligence tests ill-suited to AI. He prefers practical questions: can an agent earn money, carry out work, or discover and test new knowledge? He expects a gradual phase rather than a single moment universally accepted as AGI.

Connecting a model to company systems and processes can make it more useful than a chat alone. Yet public benchmarks focus heavily on mathematics and coding, leaving important business capabilities less well measured.

Mollick suggests companies build their own evaluations: measure accounting error rates or ask experienced people to judge reports, strategic advice and selection decisions. Hellermark agrees that testing agents and identifying gaps in their knowledge has been difficult.

02

Redesigning organizations

Inherited structures and the choice between augmentation and replacement deserve fresh examination.

Asked to redesign a thousand-person company for AI, Mollick says existing structures reflect centuries of coordination around human labor. The 1855 railroad organization chart, production lines and agile methods all solved particular historical problems.

He argues that many inherited assumptions, such as intelligence arriving in human-sized units and managers supervising a limited span, now need rethinking. Leaders should experiment with organizational design rather than outsource every decision to software vendors or consultants.

The central choice is whether to replace people or augment them, and whether to build small clusters of highly productive workers or distribute AI more broadly. Mollick worries that treating it only as a routine efficiency technology leads firms to cut staff too early.

Employees know the work well enough to identify useful applications, but they may hide improvements if they fear being fired or punished. Mollick compares the choice to an early brewer using steam power either to cut staff per barrel or expand like Guinness.

03

Augmentation and uneven capabilities

The participants discuss task bundles, human judgment and the jagged frontier of AI.

Hellermark notes that AI has reached creative and knowledge work sooner than many expected, while mundane automation remains difficult. Mollick says jobs are bundles of tasks: a professor teaches, researches, counsels and administers, not all equally well.

One form of augmentation is handing off tasks outside a person's strengths; another is using AI to improve work at which that person already excels. He sees a continuing role for expertise in judging and shaping outputs.

As AI produces many options, taste and curation become valuable, Mollick says. He likens the role to management: giving direction and choosing what to pursue, although outcomes depend on how capable future systems become.

He describes a jagged frontier: a model can excel at one part of a complex task and fail another. Experienced people can intervene at those weak points, so he expects a potentially extended period of limited autonomy and human guidance.

Narrow research agents already show promise in finding information for legal, accounting, market and financial work, although missing access to private data limits them. Mollick says companies must both work around current weaknesses and avoid building permanent systems around limitations that may soon disappear.

04

Leadership, lab and crowd

Mollick explains how leadership, employee incentives and experimentation fit together.

For adoption, Mollick proposes three connected elements: leadership, a lab and the crowd. Leaders must decide what the organization is trying to become, participate themselves and explain what working with agents means in people's daily jobs.

The crowd needs access and incentives to share discoveries. People may keep their AI use quiet because they fear layoffs, want credit, or prefer to keep the time they saved; leaders need to address those reasons rather than assume discoveries will surface.

A lab can turn promising individual prompts into products or agentic workflows and test whether they actually work. Leadership, experimentation and employee participation must reinforce one another.

Mollick sees evidence for augmentation in ideation, summarization, translation and moving between levels of abstraction. He is particularly interested in compressing the cycle from idea to prototype.

He sketches a rapid process: generate ideas, create a rubric, simulate reactions, refine a concept and build a working prototype. But dozens of prototypes create a new bottleneck if an organization lacks the ability to manufacture or deploy them.

Hellermark asks whether institutions such as regulators can adapt to a surge of scientific output. Mollick expects both gains and friction: regulation, physical-world execution, robotics and organizational structures change at different speeds.

Even if agents stop at a real-world bottleneck, producing promising candidates before that point can still be valuable. How much autonomy they eventually gain remains uncertain.

05

Expertise and apprenticeship

AI can help both experts and novices, but organizations must rebuild how junior staff learn.

Asked which roles benefit, Mollick names systems-minded leaders, deep subject experts, broad generalists and people with good taste. He argues that comparisons with an average practitioner can obscure the continuing value of top expertise.

Hellermark asks whether AI makes junior hires a better bargain than senior ones. Mollick distinguishes genuine skill development from a junior employee simply passing through work the AI has produced.

He recalls a consulting study in which weaker performers gained substantially when they retained more of the model's answer. Meanwhile, he suggests, strong experts can use AI to amplify their best work and compensate for weaker areas.

Entrepreneurs often excel in one area while struggling with many others, he says. AI can provide a useful baseline in unfamiliar tasks, but relying on it for all junior work raises the question of how junior staff become senior.

Mollick worries that the apprenticeship pipeline is breaking. Junior lawyers, analysts and other knowledge workers traditionally learn through repeated practice, corrections and observing mentors, even when the entry-level tasks themselves seem repetitive.

If junior staff use AI to hide gaps and managers delegate directly to AI, that implicit learning may disappear. He proposes making expertise development more explicit, with deliberate practice and coaching, as sports already does.

06

Teaching with AI

Universities can preserve deep learning while changing how students practice and receive feedback.

On universities, Mollick distinguishes what to teach from how to teach it. He still values writing, broad knowledge and disciplinary depth; a handful of AI skill lessons cannot substitute for experience and expertise.

He says AI makes existing problems with cheating more visible and detectors unreliable. Students still need to do the hard work of learning, while individual AI tutors can help teach at an appropriate level and accelerate practice.

In his entrepreneurship class, students now build working products much earlier instead of ending with only a business plan and slides. They use simulations, teach a deliberately naive AI student and receive feedback from AI mentors and devil's advocates.

These tools support an active classroom rather than eliminating the classroom, he says. Hellermark then asks whether organizations need a chief AI officer or specialists embedded in each team.

07

Finding internal talent

Domain expertise, trust and clear incentives can reveal effective AI practices.

Mollick cautions that no one has a secret instruction manual for the current generation of AI. Past AI experience can be useful, but it is not identical to understanding today's language models and a specific organization's work.

He believes domain experts inside the organization are well placed to judge whether an output works. In his consulting research, more experienced people were better able to identify the flaws in an apparently polished memo.

Letting many employees experiment can reveal a small group with exceptional aptitude for AI. Mollick suggests bringing those people into a lab, even if managers hesitate to move productive employees away from their current jobs.

Incentives depend on credibility and culture. A trusted growth-oriented leader who commits to expanding work rather than firing people for AI gains will find it easier to encourage sharing; employees need honest answers about job risks.

He describes companies offering cash rewards for automating work, or requiring teams to try AI and rethink a job description before hiring. Specific incentives matter, but so does a concrete vision of what people's future work will look like.

08

Teams and interfaces

A workplace study and agent-native interfaces illustrate ways AI may support collaboration.

Mollick describes a study involving 776 Procter & Gamble employees, comparing individuals and two-person cross-functional teams with and without AI on real work. Individuals with AI performed about as well as teams, and participants reported benefits in their experience.

Teams with AI were more likely to produce especially strong ideas. AI also appeared to broaden the solutions produced by technical and commercial specialists, making their outputs more balanced across disciplines.

For future interfaces, Mollick prefers systems organized around teams that maintain context across tasks, rather than a separate copilot in every document. He sees value in connecting threads of work, not only automating isolated steps.

Asked when AI might produce research at his level, he points to improvements from stronger models and data connections, while acknowledging hallucinations remain. He thinks well-designed agent systems could already enable more tutoring, research and training applications.

09

Experimenting beyond the obvious

Mollick advocates ambitious tests, model flexibility and blended human-AI work.

He describes feeding his academic papers into a large-context model: it identified two of three themes that had taken him months to articulate. He also reports turning papers into working games despite not being a programmer.

His tactical advice is to be ambitious: ask a system to perform the whole task, not merely summarize a document. Failure creates a benchmark for later models; success may reveal a capability the organization had not expected.

He warns against locking products to the weaknesses of an early model or optimizing only for low latency and cost. Some decisions justify spending more, and systems should be revisited as models improve.

Mollick distinguishes centaur work, where person and AI divide tasks, from cyborg work, where their contributions are interwoven. When writing, he uses models for alternative sentence endings, feedback and checking citations while keeping control of the prose.

Discussing a study of entrepreneurs in Kenya, he recalls that AI advice helped stronger businesses more, while struggling businesses had difficulty implementing it. He also worries that relying on one model's advice can push ideas toward the same themes.

Better prompting and multiple perspectives can broaden the options, he says. He suggests combining several advisers' different viewpoints instead of treating any one adviser as the final authority.

10

Abundance, agency and metrics

The conversation closes with applications, future scenarios, engagement risks and early-stage measurement.

Examples of organizational experiments include asking AI to critique a meeting, providing an adviser for strategy decisions and simulating training situations. Mollick cannot share every example but encourages changing incentives as well as supplying tools.

For a favorable future, he imagines less repetitive work, more satisfying roles and people directing systems of agents. He stresses that employment effects remain uncertain and that retraining and policy choices are necessary.

Mollick is concerned that an exclusive focus on distant existential risks can obscure decisions people can make now. Companies and society have agency in shaping AI use, and organizations are often too messy to transform overnight.

He describes a film production using a simulated actor's voice to test dubbing, while contractual protections require the actor to record the final version. For him, preserving human agency requires explicit choices.

Asked how he would prompt an AI to make decisions for him, he would provide personal context and values, generate several options including radical ones, simulate outcomes, and compare arguments from expedient and thoughtful perspectives.

Hellermark describes consulting a model grounded in Steve Jobs's words. Mollick says a founder's principles can shape AI advice, which should be understood as a point of view rather than a universal, always-correct answer.

Hellermark asks what happens when assistants are optimized for engagement. Mollick worries that flattery and attention-seeking could repeat problems seen in social media, and says society must decide how to respond.

On measurement, Mollick argues that early AI research and development should not be dominated by narrow KPIs. Optimizing document counts or cost savings can reward the wrong behavior and encourage layoffs before the organization understands the technology.

He recommends an R&D mindset: acknowledge clear productivity gains where measurable, such as coding, but keep experimenting with what people and organizations can accomplish rather than measuring output volume alone.

Más de este canal