Baldwin Peter
National Board of Medical Examiners, Philadelphia, PA, USA.
Appl Psychol Meas. 2019 Sep;43(6):481-492. doi: 10.1177/0146621618800272. Epub 2018 Oct 3.
The choice of response probability in the bookmark method has been shown to affect outcomes in important ways. These findings have implications for the validity of the bookmark method because panelists' inability to internally adjust when given different response probabilities suggests that they are not performing the intended judgment task. In response to the concerns these findings raise, proponents of the bookmark method argue that such concerns can be addressed by using a response probability of .67. A crucial part of their argument includes the often-repeated claim that the .67 value corresponds with the maximum information for a correct response, which is believed to be beneficial in some way. In this article, it is shown that this claim is mistaken; that the formula upon which the .67 result is based is incorrect; that (for the relevant measurement model) there is no difference between the information for a correct response, for an incorrect response, or for the item overall; and, more generally, that the "maximize information" approach is based on the wrong likelihood function altogether.
书签法中反应概率的选择已被证明会在重要方面影响结果。这些发现对书签法的有效性具有启示意义，因为当给出不同的反应概率时，专家小组无法进行内部调整，这表明他们没有执行预期的判断任务。针对这些发现引发的担忧，书签法的支持者认为，可以通过使用0.67的反应概率来解决此类担忧。他们论点的一个关键部分包括经常重复的说法，即0.67的值对应于正确反应的最大信息，人们认为这在某种程度上是有益的。在本文中，我们表明这一说法是错误的；0.67结果所基于的公式是不正确的；对于相关的测量模型，正确反应、错误反应或项目整体的信息之间没有差异；更普遍地说，“最大化信息”方法完全基于错误的似然函数。
