Evolving general cooperation with a Bayesian theory of mind.

Theories of the evolution of cooperation through reciprocity explain how unrelated self-interested individuals can accomplish more together than they can on their own. The most prominent theories of reciprocity, such as tit-for-tat or win-stay-lose-shift, are inflexible automata that lack a theory of mind-the human ability to infer the hidden mental states in others' minds. Here, we develop a model of reciprocity with a theory of mind, the Bayesian Reciprocator. When making decisions, this model does not simply seek to maximize its own payoff. Instead, it also values the payoffs of others-but only to the extent it believes that those others are also cooperating in the same way. To compute its beliefs about others, the Bayesian Reciprocator uses a probabilistic and generative approach to infer the latent preferences, beliefs, and strategies of others through interaction and observation. We evaluate the Bayesian Reciprocator using a generator over games where every interaction is unique, as well as in classic environments such as the iterated prisoner's dilemma. The Bayesian Reciprocator enables the emergence of both direct-reciprocity when games are repeated and indirect-reciprocity when interactions are one-shot but observable to others. In an evolutionary competition, the Bayesian Reciprocator outcompetes existing automata strategies and sustains cooperation across a larger range of environments and noise settings than prior approaches. This work quantifies the advantage of a theory of mind for cooperation in an evolutionary game theoretic framework and suggests avenues for building artificially intelligent agents with more human-like learning mechanisms that can cooperate across many environments.