Perinatal artificial intelligence in ultrasound (PAIR) study: predicting delivery timing.

OBJECTIVE

To evaluate the ability of a proprietary artificial intelligence (AI) model to predict the number of days until delivery using ultrasound images alone and to assess the continuous improvement of prediction accuracy, particularly for preterm births, through model retraining.

METHODS

An AI software was developed and trained using de-identified ultrasound images from a cohort of women who delivered at the University of Kentucky from 2017 to 2021. Initially, 5,714 pregnant women, with 19,940 unique ultrasound exams and 877,141 total ultrasound images were utilized from this timeframe. Images from 79% of this cohort (4,505 patients) trained the AI to estimate the number of days until delivery and secondarily optimize predictions related to preterm birth (<37 weeks gestational age). The output consisted of days until delivery which was subsequently categorized as either preterm or term birth.The remaining 21% of the cohort (1,209 patients) was reserved for derivation and validation of test characteristics. Delivery outcomes for this subgroup were blinded from the AI by an independent third-party data monitor. Unique predictions were made for each patient after each ultrasound exam, and the AI's performance was evaluated against the actual delivery date using metrics such as R values and mean absolute error (MAE) compared to actual days until delivery. After initial testing, the AI was retrained x3 more using the same data (Version 2, V2) and later with an additional 1,165,618 images obtained by extension of the study to include data from our center until 2023 (Version 3 (V3), Version 4 (V4)- consisted of retraining on V3).

RESULTS

Preterm birth rates were similar between the training (18.4%) and validation (18.6%) sets in the initial study set. The initial AI model exhibited a sensitivity of 39% and specificity of 93% for preterm birth prediction, with an AUC of 0.757. The AI's predictions of days to delivery versus actual in the validation set yielded R of 0.90 for term births, 0.88 for spontaneous preterm birth plus term births, and 0.48 for spontaneous preterm birth alone. The MAE in predicting the number of days until delivery showed similar accuracy across all trimesters that were assessed by image analysis. Finally, retraining with improvements in AI architecture and training methodology using additional images provided improved preterm birth prediction, with R values for all births increasing from 0.85 (V1) to 0.88 (V3) to 0.92 (V4). For spontaneous PTB, MAE was 19.99 days in V4.

CONCLUSIONS

AI can predict timing until delivery from ultrasound data alone. This technology can also predict preterm delivery with limited sensitivity. Retraining the AI with supervised and unsupervised learning has the potential to further improve performance.