Fazel Seena, Sariaslan Amir, Fanshawe Thomas
Department of Psychiatry, University of Oxford, Warneford Hospital, Oxford, UK.
Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK.
Eur J Crim Pol Res. 2022;28(3):397-406. doi: 10.1007/s10610-022-09520-y. Epub 2022 Jul 1.
Risk assessment tools are widely used throughout the criminal justice system to assist in making decisions about sentencing, supervision, and treatment. In this article, we discuss several methodological and practical limitations associated with risk assessment tools currently in use. These include variable predictive performance due to the exclusion of important background predictors; high costs, including the need for regular staff training, in order to use many tools; development of tools using suboptimal methods and poor transparency in how they create risk scores; included risk factors being based on dated evidence; and ethical concerns highlighted by legal scholars and criminologists, such as embedding systemic biases and uncertainty about how these tools influence judicial decisions. We discuss the potential that specific predictors, such as living in a deprived neighbourhood, may indirectly select for individuals in racial or ethnic minority groups. To demonstrate how these limitations and ethical concerns can be addressed, we present the example of OxRec, a risk assessment tool used to predict recidivism for individuals in the criminal justice system. OxRec was developed in Sweden and has been externally validated in Sweden and the Netherlands. The advantages of OxRec include its predictive accuracy based on rigorous multivariable testing of predictors, transparent reporting of results and the final model (including how the probability score is derived), scoring simplicity (i.e. without the need for additional interview), and the reporting of a wide range of performance measures, including those of discrimination and calibration, the latter of which is rarely reported but a key metric. OxRec is intended to be used alongside professional judgement, as a support for decision-making, and its performance measures need to be interpreted in this light. The reported calibration of the tool in external samples clearly suggests no systematic overestimation of risk, including in large subgroups.
风险评估工具在整个刑事司法系统中被广泛使用，以协助做出关于量刑、监管和治疗的决策。在本文中，我们讨论了与当前使用的风险评估工具相关的几个方法和实际限制。这些限制包括：由于排除了重要的背景预测因素，预测性能存在差异；使用许多工具的成本高昂，包括定期员工培训的需求；工具开发方法欠佳，创建风险评分的过程缺乏透明度；纳入的风险因素基于过时的证据；以及法律学者和犯罪学家强调的伦理问题，例如嵌入系统性偏见以及这些工具如何影响司法决策存在不确定性。我们讨论了特定预测因素（如生活在贫困社区）可能间接选择种族或少数民族群体个体的可能性。为了说明如何解决这些限制和伦理问题，我们举了OxRec的例子，这是一种用于预测刑事司法系统中个体再犯风险的评估工具。OxRec在瑞典开发，并已在瑞典和荷兰进行了外部验证。OxRec的优点包括：基于对预测因素的严格多变量测试的预测准确性、结果和最终模型的透明报告（包括概率得分是如何得出的）、评分简单（即无需额外面谈），以及报告广泛的性能指标，包括歧视和校准指标，后者很少被报告，但却是关键指标。OxRec旨在与专业判断一起使用，作为决策的支持，其性能指标需要据此进行解释。该工具在外部样本中的校准报告清楚地表明没有系统性的风险高估，包括在大型亚组中。
