Boyd Chris, Kleinig Timothy J, Dawson Joseph, Patel Sandy, Mayer Wolfgang, Bezak Eva
Allied Health and Human Performance, University of South Australia, Adelaide, Australia.
Medical Physics and Radiation Safety, South Australia Medical Imaging, Adelaide, Australia.
J Neuroimaging. 2025 May-Jun;35(3):e70058. doi: 10.1111/jon.70058.
This work evaluates carotid atherosclerosis quantification from computed tomography angiography (CTA), by novice and expert human contours. Variability sources are critically assessed to establish the minimum performance of future machine learning (ML) tools.
We analyzed extra cranial carotid lesions, with no, mild, moderate, and severe atherosclerosis (n = 10/group). CTA datasets of 24 patients (n = 6/group) were re-sampled to 2.5 mm axial thicknesses. Lumen, calcific plaque, and soft plaque were manually contoured by three expert experienced clinicians (neuroradiologist, vascular neurologist, and vascular surgeon), a medical physicist (MP), and a radiographer. Contouring was repeated several months later for intra-operator variability and again after development of a protocol. Clinicians blindly ranked each other's contours for descriptive statistical analysis.
Relative to internal carotid origin, plaque began a median of 3.75 mm inferior (Interquartile Range [IQR] 0.8-7 mm), extended 18 mm superior (IQR: 13.0-29.6 mm), with a median total length of 24.4 mm (IQR: 14.7-37.4 mm). Clinicians and non-clinicians contoured lumen and calcific plaque similarly (dice similarity coefficient [DSC]: 0.87/0.62 respectively), but varied greater for soft plaque (DSC: 0.21). Neuroradiologist contours were consistently smaller, from approaching the partial-volume artifact conservatively. Clinicians favored their own contours, most pronouncedly the neuroradiologist (standard deviation: 0.00). Establishing a contouring protocol was not found to improve the agreement between clinicians.
CTA carotid pathology contouring inherently has limited clinician agreement due to small structure size and poor contrast. The reference-contour datasets produced by experienced clinicians are prone to inter-and intra-variability which must be carefully considered to ensure ML models developed from such datasets are not fatally flawed.
本研究通过新手和专家手动勾勒轮廓，评估计算机断层血管造影（CTA）对颈动脉粥样硬化的量化。对变异性来源进行了严格评估，以确定未来机器学习（ML）工具的最低性能。
我们分析了无动脉粥样硬化、轻度、中度和重度动脉粥样硬化的颅外颈动脉病变（每组10例）。将24例患者（每组6例）的CTA数据集重新采样至2.5mm轴向厚度。由三位经验丰富的临床医生（神经放射科医生、血管神经科医生和血管外科医生）、一名医学物理学家（MP）和一名放射技师手动勾勒管腔、钙化斑块和软斑块的轮廓。几个月后重复勾勒轮廓以评估操作者内部的变异性，并在制定方案后再次进行。临床医生对彼此的轮廓进行盲法排序，以进行描述性统计分析。
相对于颈内动脉起始处，斑块起始于下方中位数为3.75mm（四分位间距[IQR]0.8 - 7mm），向上延伸18mm（IQR：13.0 - 29.6mm），总长度中位数为24.4mm（IQR：14.7 - 37.4mm）。临床医生和非临床医生对管腔和钙化斑块的勾勒相似（骰子相似系数[DSC]分别为0.87/0.62），但对软斑块的差异更大（DSC：0.21）。神经放射科医生勾勒的轮廓始终较小，这是因为他们保守地接近部分容积伪影。临床医生更倾向于自己勾勒的轮廓，神经放射科医生最为明显（标准差：0.00）。未发现制定轮廓勾勒方案能改善临床医生之间的一致性。
由于结构尺寸小和对比度差，CTA颈动脉病变轮廓勾勒在临床医生之间的一致性固有地有限。经验丰富的临床医生生成的参考轮廓数据集容易出现个体间和个体内的变异性，必须仔细考虑这一点，以确保从此类数据集开发的ML模型没有致命缺陷。
