Department of Computer Science & Centre for Medical Image Computing, University College London, London, United Kingdom.
Centre for Medical Imaging, University College London, London, United Kingdom.
PLoS One. 2021 Oct 8;16(10):e0258442. doi: 10.1371/journal.pone.0258442. eCollection 2021.
This paper proposes a task-driven computational framework for assessing diffusion MRI experimental designs which, rather than relying on parameter-estimation metrics, directly measures quantitative task performance. Traditional computational experimental design (CED) methods may be ill-suited to experimental tasks, such as clinical classification, where outcome does not depend on parameter-estimation accuracy or precision alone. Current assessment metrics evaluate experiments' ability to faithfully recover microstructural parameters rather than their task performance. The method we propose addresses this shortcoming. For a given MRI experimental design (protocol, parameter-estimation method, model, etc.), experiments are simulated start-to-finish and task performance is computed from receiver operating characteristic (ROC) curves and associated summary metrics (e.g. area under the curve (AUC)). Two experiments were performed: first, a validation of the pipeline's task performance predictions against clinical results, comparing in-silico predictions to real-world ROC/AUC; and second, a demonstration of the pipeline's advantages over traditional CED approaches, using two simulated clinical classification tasks. Comparison with clinical datasets validates our method's predictions of (a) the qualitative form of ROC curves, (b) the relative task performance of different experimental designs, and (c) the absolute performance (AUC) of each experimental design. Furthermore, we show that our method outperforms traditional task-agnostic assessment methods, enabling improved, more useful experimental design. Our pipeline produces accurate, quantitative predictions of real-world task performance. Compared to current approaches, such task-driven assessment is more likely to identify experimental designs that perform well in practice. Our method is not limited to diffusion MRI; the pipeline generalises to any task-based quantitative MRI application, and provides the foundation for developing future task-driven end-to end CED frameworks.
本文提出了一种基于任务的计算框架，用于评估扩散 MRI 实验设计，该框架不依赖于参数估计指标，而是直接测量定量任务性能。传统的计算实验设计 (CED) 方法可能不适合于某些实验任务，例如临床分类，其结果不仅取决于参数估计的准确性和精度。当前的评估指标评估的是实验重现微观结构参数的能力，而不是它们的任务性能。我们提出的方法解决了这一缺点。对于给定的 MRI 实验设计（方案、参数估计方法、模型等），我们从头到尾模拟实验，并从接收者操作特性（ROC）曲线及其相关的汇总指标（例如曲线下面积（AUC））计算任务性能。进行了两项实验：首先，将该方法对任务性能的预测与临床结果进行了验证，将仿真预测与实际 ROC/AUC 进行了比较；其次，使用两个模拟的临床分类任务，展示了该方法相对于传统 CED 方法的优势。与临床数据集的比较验证了我们方法的预测：（a）ROC 曲线的定性形式，（b）不同实验设计的相对任务性能，以及（c）每个实验设计的绝对性能（AUC）。此外，我们表明我们的方法优于传统的无任务评估方法，从而能够实现更好、更有用的实验设计。我们的方法可以准确地预测真实世界任务性能的定量指标。与当前的方法相比，这种基于任务的评估更有可能识别在实践中表现良好的实验设计。我们的方法不仅限于扩散 MRI；该方法适用于任何基于任务的定量 MRI 应用，并为开发未来基于任务的端到端 CED 框架奠定了基础。
