Tuning your priors to the world.

The idea that perceptual and cognitive systems must incorporate knowledge about the structure of the environment has become a central dogma of cognitive theory. In a Bayesian context, this idea is often realized in terms of "tuning the prior"-widely assumed to mean adjusting prior probabilities so that they match the frequencies of events in the world. This kind of "ecological" tuning has often been held up as an ideal of inference, in fact defining an "ideal observer." But widespread as this viewpoint is, it directly contradicts Bayesian philosophy of probability, which views probabilities as degrees of belief rather than relative frequencies, and explicitly denies that they are objective characteristics of the world. Moreover, tuning the prior to observed environmental frequencies is subject to overfitting, meaning in this context overtuning to the environment, which leads (ironically) to poor performance in future encounters with the same environment. Whenever there is uncertainty about the environment-which there almost always is-an agent's prior should be biased away from ecological relative frequencies and toward simpler and more entropic priors.