Quality of radiomics research: comprehensive analysis of 1574 unique publications from 89 reviews.

PURPOSE

This study aims to comprehensively evaluate the quality of radiomics research by examining unique papers from reviews using the radiomics quality score (RQS).

METHODS

A literature search was conducted in PubMed (last search date: April 14, 2024). Systematic or non-systematic reviews using the RQS to evaluate radiomic studies were potentially included. Exclusion was applied at two levels: first, at the review level, and second, at the study level (i.e., for the individual articles previously evaluated within the reviews). Score-wise and item-wise analyses were performed, along with trend, multivariable, and subgroup analyses based on baseline study characteristics and validation methods.

RESULTS

A total of 1574 unique papers (published online between 1999 and 2023) from 89 reviews were included in the final analysis. The median RQS percentage was 31% with an IQR of 25% (25th-75th percentiles, 14-39%). A positive correlation between median RQS percentage and publication year (2014-2023) was found, with Kendall's tau coefficient of 0.908 (p < 0.001), suggesting an improvement in quality over time. The quality of radiomics publications significantly varied according to different subfields of radiology (p < 0.001). Around one-third of the publications (32%) lacked a separate validation set. Papers with internal validation (54%) dominated those with external validation (14%). Higher-quality validation practices were significantly associated with better RQS percentage scores, independent of the validation's effect on the final score. Item-wise analysis revealed significant shortcomings in several areas.

CONCLUSION

Radiomics research quality is low but improving according to RQS. Significant variation exists across radiology subfields. Critical areas were identified for targeted improvement.

CLINICAL RELEVANCE STATEMENT

Our study shows that the quality of radiomics research is generally low but improving over time, with item-wise analysis highlighting critical areas needing improvement. It also reveals that the quality of radiomics research differs across subfields and validation methods.

KEY POINTS

Overall quality of radiomics research remains low and highly variable, although a significant positive trend suggests an improvement in quality over time. Considerable variations exist in the quality of radiomics publications across different subfields of radiology and validation types. The item-wise analysis highlights several critical areas requiring attention, emphasizing the need for targeted improvements.