Neural Systems Laboratory, Department of Health and Rehabilitation Sciences, Boston University, Boston, MA, USA
Irrational Agency, London, UK
Behav Brain Sci. 2023 Jun 26;47:e67. doi: 10.1017/S0140525X23002753.
When a measure becomes a target, it ceases to be a good measure. For example, when standardized test scores in education become targets, teachers may start "teaching to the test," leading to breakdown of the relationship between the measure - test performance - and the underlying goal - quality education. Similar phenomena have been named and described across a broad range of contexts, such as economics, academia, machine learning, and ecology. Yet it remains unclear whether these phenomena bear only superficial similarities, or if they derive from some fundamental unifying mechanism. Here, we propose such a unifying mechanism, which we label . We first review illustrative examples and their labels, such as the "cobra effect," "Goodhart's law," and "Campbell's law." Second, we identify central prerequisites and constraints of proxy failure, noting that it is often only a partial failure or . We argue that whenever incentivization or selection is based on an imperfect proxy measure of the underlying goal, a pressure arises that tends to make the proxy a worse approximation of the goal. Third, we develop this perspective for three concrete contexts, namely neuroscience, economics, and ecology, highlighting similarities and differences. Fourth, we outline consequences of proxy failure, suggesting it is key to understanding the structure and evolution of goal-oriented systems. Our account draws on a broad range of disciplines, but we can only scratch the surface within each. We thus hope the present account elicits a collaborative enterprise, entailing both critical discussion as well as extensions in contexts we have missed.
当一项措施成为目标时，它就不再是一个好的措施。例如，当教育中的标准化考试成绩成为目标时，教师可能会开始“应试教学”，导致衡量标准——考试成绩与基本目标——优质教育之间的关系破裂。在经济学、学术界、机器学习和生态学等广泛的背景下，都已经出现了类似的现象，并对其进行了描述和命名。然而，目前还不清楚这些现象是否只是表面上的相似，还是源于某种基本的统一机制。在这里，我们提出了这样一种统一机制，我们称之为“代理失败”。首先，我们回顾了一些具有代表性的例子及其标签，例如“眼镜蛇效应”、“古德哈特定律”和“坎贝尔定律”。其次，我们确定了代理失败的核心前提和约束条件，并指出它通常只是部分失败或不完全失败。我们认为，只要激励或选择是基于对基本目标的不完善代理措施，就会产生一种压力，使代理措施更难以准确反映目标。第三，我们为神经科学、经济学和生态学这三个具体领域发展了这一观点，强调了它们之间的相似性和差异性。第四，我们概述了代理失败的后果，认为它是理解目标导向系统结构和演化的关键。我们的论述借鉴了广泛的学科，但在每个学科中我们只能触及皮毛。因此，我们希望本报告能引发一个协作的事业，包括批判性的讨论以及在我们忽略的背景下的扩展。
