Abstract
Background
Youth who exit the nation’s foster care system without permanency are at high risk of experiencing difficulties during the transition to adulthood.
Objective
To present an illustrative test of whether an algorithmic decision aid could be used to identify youth at risk of exiting foster care without permanency.
Methods
For youth placed in foster care between ages 12 and 14, we assessed the risk of exiting care without permanency by age 18 based on their child welfare service involvement history. To develop predictive risk models, 28 years (1991–2018) of child welfare service records from California were used. Performances were evaluated using F1, AUC, and precision and recall scores at k %. Algorithmic racial bias and fairness was also examined.
Results
The gradient boosting decision tree and random forest showed the best performance (F1 score = .54–.55, precision score = .62, recall score = .49). Among the top 30 % of youth the model identified as high risk, half of all youth who exited care without permanency were accurately identified four to six years prior to their exit, with a 39 % error rate. Although racial disparities between Black and White youth were observed in imbalanced error rates, calibration and predictive parity were satisfied.
Conclusions
Our study illustrates the manner in which potential applications of predictive analytics, including those designed to achieve universal goals of permanency through more targeted allocations of resources, can be tested. It also assesses the model using metrics of fairness.