Gao Riqiang, Tang Yucheng, Xu Kaiwen, Huo Yuankai, Bao Shunxing, Antic Sanja L, Epstein Emily S, Deppen Steve, Paulson Alexis B, Sandler Kim L, Massion Pierre P, Landman Bennett A
Electrical Engineering and Computer Science, Vanderbilt University, Nashville, TN 37235, USA.
Electrical Engineering and Computer Science, Vanderbilt University, Nashville, TN 37235, USA.
Med Image Anal. 2020 Oct;65:101785. doi: 10.1016/j.media.2020.101785. Epub 2020 Jul 18.
The Long Short-Term Memory (LSTM) network is widely used in modeling sequential observations in fields ranging from natural language processing to medical imaging. The LSTM has shown promise for interpreting computed tomography (CT) in lung screening protocols. Yet, traditional image-based LSTM models ignore interval differences, while recently proposed interval-modeled LSTM variants are limited in their ability to interpret temporal proximity. Meanwhile, clinical imaging acquisition may be irregularly sampled, and such sampling patterns may be commingled with clinical usages. In this paper, we propose the Distanced LSTM (DLSTM) by introducing time-distanced (i.e., time distance to the last scan) gates with a temporal emphasis model (TEM) targeting at lung cancer diagnosis (i.e., evaluating the malignancy of pulmonary nodules). Briefly, (1) the time distance of every scan to the last scan is modeled explicitly, (2) time-distanced input and forget gates in DLSTM are introduced across regular and irregular sampling sequences, and (3) the newer scan in serial data is emphasized by the TEM. The DLSTM algorithm is evaluated with both simulated data and real CT images (from 1794 National Lung Screening Trial (NLST) patients with longitudinal scans and 1420 clinical studied patients). Experimental results on simulated data indicate the DLSTM can capture families of temporal relationships that cannot be detected with traditional LSTM. Cross-validation on empirical CT datasets demonstrates that DLSTM achieves leading performance on both regularly and irregularly sampled data (e.g., improving LSTM from 0.6785 to 0.7085 on F1 score in NLST). In external-validation on irregularly acquired data, the benchmarks achieved 0.8350 (CNN feature) and 0.8380 (with LSTM) on AUC score, while the proposed DLSTM achieves 0.8905. In conclusion, the DLSTM approach is shown to be compatible with families of linear, quadratic, exponential, and log-exponential temporal models. The DLSTM can be readily extended with other temporal dependence interactions while hardly increasing overall model complexity.
长短期记忆（LSTM）网络在从自然语言处理到医学成像等领域的序列观测建模中得到了广泛应用。LSTM在肺部筛查方案中解读计算机断层扫描（CT）方面显示出了前景。然而，传统的基于图像的LSTM模型忽略了时间间隔差异，而最近提出的基于时间间隔建模的LSTM变体在解释时间接近度方面能力有限。同时，临床成像采集可能是不规则采样的，并且这种采样模式可能与临床应用混合在一起。在本文中，我们通过引入具有时间距离（即到最后一次扫描的时间距离）门的距离LSTM（DLSTM），并结合针对肺癌诊断（即评估肺结节的恶性程度）的时间强调模型（TEM）。简而言之，（1）明确建模每次扫描到最后一次扫描的时间距离，（2）在DLSTM中引入跨常规和不规则采样序列的时间距离输入门和遗忘门，（3）通过TEM强调序列数据中较新的扫描。使用模拟数据和真实CT图像（来自1794名接受纵向扫描的国家肺部筛查试验（NLST）患者和1420名临床研究患者）对DLSTM算法进行了评估。模拟数据实验结果表明，DLSTM能够捕捉传统LSTM无法检测到的时间关系族。在经验性CT数据集上的交叉验证表明，DLSTM在常规和不规则采样数据上均取得了领先性能（例如，在NLST中，F1分数从0.6785提高到0.7085）。在对不规则采集数据的外部验证中，基准在AUC分数上达到了0.8350（CNN特征）和0.8380（使用LSTM），而所提出的DLSTM达到了0.8905。总之，DLSTM方法被证明与线性、二次、指数和对数指数时间模型族兼容。DLSTM可以很容易地与其他时间依赖相互作用扩展，而几乎不增加整体模型复杂性。
