When Should AI Agents Cooperate?

Two participants consider mutual benefit, conflicting incentives and the line between cooperation and collusion.

Channel
Cognitive Revolution "How AI Changes Everything"
Published
Feb 13, 2025
Video
1:08
Read
1 min
Original
English
Open on YouTube
Scroll to read and the video shrinks to the bottom-right corner. Captions follow your reading language.
Reading languageAvailable in 2 languages
You're reading the original English.
SUMMARY

In 30 seconds

In this short discussion, two participants ask how AI agents should act when their human principals have different interests. They contrast beneficial cooperation with undesirable collusion; the original video has embedding disabled.

  1. 1What should AI cooperate toward?The speakers ask how AI agents should behave when their human principals have conflicting incentives.
  2. 2Mutual benefitAgents acting for different people or organizations may cooperate for shared benefit.
  3. 3The cooperation–collusion distinctionThe second speaker warns that price collusion is an example where cooperation is undesirable; context matters.

Takeaways

  1. The speakers ask how AI agents should behave when their human principals have conflicting incentives.
  2. Agents acting for different people or organizations may cooperate for shared benefit.
  3. The second speaker warns that price collusion is an example where cooperation is undesirable; context matters.
FULL TEXT

Read the edited text by chapter

Transcript only
01

The question of cooperation

The speakers ask how AI agents should behave when their human principals have conflicting incentives.

One participant says that a positive vision of the future is scarce and asks what people should want AI to do. How can we reach good outcomes when humans may want to defect, or ask an AI to defect on their behalf?

02

Benefits and risks

Agents acting for different people or organizations may cooperate for shared benefit. The second speaker warns that price collusion is an example where cooperation is undesirable; context matters.

The other participant replies that interactions among agents will take many forms. In some settings, agents representing individuals or organizations can cooperate to reach outcomes that benefit both sides; in those cases, we want them to succeed at cooperation.

In other situations, however, we do not want AIs to cooperate in ways such as colluding on prices. Whether cooperation is appropriate depends on the details. The first participant notes that the line between cooperation and collusion can depend on perspective, and the other agrees. The original video has embedding disabled.

More from this channel

1:49:40EN
Cognitive Revolution "How AI Changes Everything"· Oct 28, 2026Podcast · 4 speakersA Strategy for Preserving Options on the Path to Advanced AIGuest Joel Predd, Ben Boudreaux, Matt Chessen · Host Nathan LabenzRAND researchers discuss uncertainty, security architecture and human agency with Nathan Labenz.AI StrategyGeopoliticsAI Safety+1
1:28:36EN
Cognitive Revolution "How AI Changes Everything"· Oct 24, 2026Podcast · 5 speakersAI’s Pace: Interpretability, GPU Credit and the Physical Supply ChainGuest Zvi Mowshowitz, Evan Conrad, Lonis Hamaili · Host Nathan Labenz, Prakash NarayananA weekly highlights episode weighs uncertain AI research trajectories against finance, infrastructure, security and regulation.AI researchAI SafetyAI Regulation
3:13:44EN
Cognitive Revolution "How AI Changes Everything"· Oct 22, 2026Podcast · 4 speakersAI’s Physical Bottlenecks: Compute Markets, Chip Investments and Agent TrustGuest Evan Conrad, Lonis Hamaili · Host Nathan Labenz, Prakash NarayananA full AI:AM discussion of security incidents, GPU financing, semiconductor supply and an agent-to-agent network.Infrastructure & deploymentAI securityInvesting+2
2:40:58EN
Cognitive Revolution "How AI Changes Everything"· Oct 21, 2026Podcast · 3 speakersFrontier AI, Interpretability and the Fight Over Its PaceGuest Zvi Mowshowitz · Host Nathan Labenz, Prakash NarayananZvi Mowshowitz joins AI:AM to discuss neural networks, policy and the institutions facing rapid change.AI SafetyAI RegulationAI research+2
1:25:56EN
Cognitive Revolution "How AI Changes Everything"· Oct 17, 2026PodcastClaude Changed the Password. The Steering Wheel Was Already OptionalHost Nathan LabenzNathan Labenz and Prakash Narayanan spend a week on agents that can lock you out, models that dodge a feeling they deny having, and formal proofs that found Linux bugs.AgentsSafety & governanceAgent engineering+3
3 slides2:51:22EN
Cognitive Revolution "How AI Changes Everything"· Oct 14, 2026Podcast · 4 speakersAI Minds and 8 mm Memories: Research, Archives and SimulationGuest Cameron Berg, Phil Maher · Host Nathan Labenz, Prakash NarayananCameron Berg discusses hidden model states; Phil Maher explains preserving home movies and a fly-brain demo.ConsciousnessAI researchAI Safety+2