Evolution of state-dependent strategies in stochastic games.

In a population of interacting individuals, the environment for interactions often changes due to individuals' behaviors, which in turn drive the evolution of individuals' behaviors. The interplay between the environment and individuals' behaviors has been demonstrated to remarkably influence the evolutionary outcomes. In reality, in highly cognitive species such as social primates and human beings, individuals are often capable of perceiving the environment change and then differentiate their strategies across different environment states. We propose a model of environmental feedback with state-dependent strategies: individuals have perceptions of distinct environment states and therefore take distinct sub-strategies under each of them; based on the sub-strategy, individuals then decide their behaviors; their behaviors subsequently modify the environment state. We use the theory of stochastic games and evolutionary dynamics to analyze this idea. We find that when environment changes slower than behaviors, state-dependent strategies (i.e. taking different sub-strategies under different environment states) can outperform state-independent strategies (i.e. taking an identical sub-strategy under all environment states), such as Win-Stay, Lose-Shift, the most leading strategy among state-independent strategies. The intuition is that delayed environmental feedback provides chances for individuals with state-dependent strategies to exploit those with state-independent strategies. Our results hold (1) in both well-mixed and structured populations; (2) when the environment switches between two or more states. Furthermore, the environment changing rate decides if state-dependent strategies benefit global cooperation. The evolution sees the rise of the cooperation level for fast environment switching and the decrease otherwise. Our work stresses that individuals' perceptions of different environment states are beneficial to their survival and social prosperity in a changing world.