When Should AI Agents Cooperate?

RÉSUMÉ

En 30 secondes

In this short discussion, two participants ask how AI agents should act when their human principals have different interests. They contrast beneficial cooperation with undesirable collusion; the original video has embedding disabled.

1What should AI cooperate toward?The speakers ask how AI agents should behave when their human principals have conflicting incentives.
2Mutual benefitAgents acting for different people or organizations may cooperate for shared benefit.
3The cooperation–collusion distinctionThe second speaker warns that price collusion is an example where cooperation is undesirable; context matters.

À retenir

The speakers ask how AI agents should behave when their human principals have conflicting incentives.
Agents acting for different people or organizations may cooperate for shared benefit.
The second speaker warns that price collusion is an example where cooperation is undesirable; context matters.

ThèmesMulti-agent systems AI Safety

TEXTE INTÉGRAL

Lire le texte remanié, chapitre par chapitre

Afficher la transcription

Transcription seule

The question of cooperation

The speakers ask how AI agents should behave when their human principals have conflicting incentives.

One participant says that a positive vision of the future is scarce and asks what people should want AI to do. How can we reach good outcomes when humans may want to defect, or ask an AI to defect on their behalf?

Benefits and risks

Agents acting for different people or organizations may cooperate for shared benefit. The second speaker warns that price collusion is an example where cooperation is undesirable; context matters.

The other participant replies that interactions among agents will take many forms. In some settings, agents representing individuals or organizations can cooperate to reach outcomes that benefit both sides; in those cases, we want them to succeed at cooperation.

In other situations, however, we do not want AIs to cooperate in ways such as colluding on prices. Whether cooperation is appropriate depends on the details. The first participant notes that the line between cooperation and collusion can depend on perspective, and the other agrees. The original video has embedding disabled.