Schwabe Daniel, Becker Katinka, Seyferth Martin, Klaß Andreas, Schaeffter Tobias
Division Medical Physics and Metrological Information Technology, Physikalisch-Technische Bundesanstalt, Berlin, Germany.
Department of Medical Engineering, Technical University Berlin, Berlin, Germany.
NPJ Digit Med. 2024 Aug 3;7(1):203. doi: 10.1038/s41746-024-01196-4.
The adoption of machine learning (ML) and, more specifically, deep learning (DL) applications into all major areas of our lives is underway. The development of trustworthy AI is especially important in medicine due to the large implications for patients' lives. While trustworthiness concerns various aspects including ethical, transparency and safety requirements, we focus on the importance of data quality (training/test) in DL. Since data quality dictates the behaviour of ML products, evaluating data quality will play a key part in the regulatory approval of medical ML products. We perform a systematic review following PRISMA guidelines using the databases Web of Science, PubMed and ACM Digital Library. We identify 5408 studies, out of which 120 records fulfil our eligibility criteria. From this literature, we synthesise the existing knowledge on data quality frameworks and combine it with the perspective of ML applications in medicine. As a result, we propose the METRIC-framework, a specialised data quality framework for medical training data comprising 15 awareness dimensions, along which developers of medical ML applications should investigate the content of a dataset. This knowledge helps to reduce biases as a major source of unfairness, increase robustness, facilitate interpretability and thus lays the foundation for trustworthy AI in medicine. The METRIC-framework may serve as a base for systematically assessing training datasets, establishing reference datasets, and designing test datasets which has the potential to accelerate the approval of medical ML products.
Cochrane Database Syst Rev. 2022-2-1
JMIR Res Protoc. 2023-10-25
Early Hum Dev. 2020-11
J Biomed Semantics. 2023-8-31
Front Psychol. 2023-1-17
Front Digit Health. 2024-2-20
Healthcare (Basel). 2022-9-30
J Med Internet Res. 2025-8-1
Influenza Other Respir Viruses. 2025-8
J Transl Med. 2025-7-24
J Med Internet Res. 2025-6-23
AMIA Jt Summits Transl Sci Proc. 2025-6-10
J Tradit Complement Med. 2025-2-21
J Am Med Inform Assoc. 2023-9-25
J Med Internet Res. 2023-3-31
Methods Inf Med. 2023-5
Nat Commun. 2022-12-15
IEEE Trans Image Process. 2022