Children's Data Network, Suzanne Dworak-Peck School of Social Work, University of Southern California, United States.
Information Sciences Institute and Department of Computer Science, University of Southern California, United States.
Child Abuse Negl. 2021 Jul;117:105059. doi: 10.1016/j.chiabu.2021.105059. Epub 2021 May 2.
Youth who exit the nation's foster care system without permanency are at high risk of experiencing difficulties during the transition to adulthood.
To present an illustrative test of whether an algorithmic decision aid could be used to identify youth at risk of existing foster care without permanency.
For youth placed in foster care between ages 12 and 14, we assessed the risk of exiting care without permanency by age 18 based on their child welfare service involvement history. To develop predictive risk models, 28 years (1991-2018) of child welfare service records from California were used. Performances were evaluated using F1, AUC, and precision and recall scores at k %. Algorithmic racial bias and fairness was also examined.
The gradient boosting decision tree and random forest showed the best performance (F1 score = .54-.55, precision score = .62, recall score = .49). Among the top 30 % of youth the model identified as high risk, half of all youth who exited care without permanency were accurately identified four to six years prior to their exit, with a 39 % error rate. Although racial disparities between Black and White youth were observed in imbalanced error rates, calibration and predictive parity were satisfied.
Our study illustrates the manner in which potential applications of predictive analytics, including those designed to achieve universal goals of permanency through more targeted allocations of resources, can be tested. It also assesses the model using metrics of fairness.
青年在离开国家寄养系统后如果没有永久性住所，他们在成年过渡期间很可能会遇到困难。
展示一个算法决策辅助工具的实例测试，该工具可以用于识别可能在寄养中没有永久性住所的风险青年。
对于 12 至 14 岁被安置在寄养家庭中的青年，我们根据他们的儿童福利服务参与历史评估了在 18 岁前离开寄养而没有永久性住所的风险。为了开发预测风险模型，我们使用了加利福尼亚州 28 年（1991-2018 年）的儿童福利服务记录。使用 F1、AUC 和 k %的精度和召回分数来评估性能。还检查了算法的种族偏见和公平性。
梯度提升决策树和随机森林显示了最佳性能（F1 分数=.54-.55，精度分数=.62，召回分数=.49）。在模型确定为高风险的前 30%的青年中，一半以上的在没有永久性住所的情况下离开寄养的青年在离开前四到六年被准确识别，错误率为 39%。尽管在不平衡错误率中观察到了黑人和白人青年之间的种族差异，但校准和预测均等性得到了满足。
我们的研究说明了预测分析的潜在应用程序的方式，包括那些旨在通过更有针对性地分配资源来实现永久性的普遍目标的应用程序，可以通过使用公平性指标来评估模型。
