Okkaoglu Yasin, Welton Nicky J, Jones Hayley E
Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK.
BMC Med Res Methodol. 2024 Dec 5;24(1):299. doi: 10.1186/s12874-024-02432-x.
Latent class models can be used to estimate diagnostic accuracy without a gold standard test. Early studies often assumed independence between tests given the true disease state, however this can lead to biased estimates when there are inter-test dependencies. Residual correlation plots and chi-squared statistics have been commonly utilized to assess the validity of the conditional independence assumption and, when it does not hold, identify which test pairs are conditionally dependent. We aimed to assess the performance of these tools with a simulation study covering a wide range of scenarios.
We generated data sets from a model with four tests and a dependence between tests 1 and 2 within the diseased group. We varied sample size, prevalence, covariance, sensitivity and specificity, with 504 combinations of these in total, and 1000 data sets for each combination. We fitted the conditional independence model in a Bayesian framework, and reported absolute bias, coverage, and how often the residual correlation plots, and statistics indicated lack-of-fit globally or for each test pair.
Across all settings, residual correlation plots, pairwise and detected the correct correlated pair of tests only 12.1%, 10.3%, and 10.3% of the time, respectively, but incorrectly suggested dependence between tests 3 and 4 64.9%, 49.7%, and 49.5% of the time. We observed some variation in this across parameter settings, with these tools appearing to perform more as intended when tests 3 and 4 were both much more accurate than tests 1 and 2. Residual correlation plots, and statistics identified a lack of overall fit in 74.3%, 64.5% and 67.5% of models, respectively. The conditional independence model tended to overestimate the sensitivities of the correlated tests (median bias across all scenarios 0.094, 2.5th and 97.5th percentiles -0.003, 0.397) and underestimate prevalence and the specificities of the uncorrelated tests.
Residual correlation plots and chi-squared statistics cannot be relied upon to identify which tests are conditionally dependent, and also have relatively low power to detect lack of overall fit. This is important since failure to account for conditional dependence can lead to highly biased parameter estimates.
潜在类别模型可用于在没有金标准测试的情况下估计诊断准确性。早期研究通常假设在给定真实疾病状态下测试之间相互独立，然而，当测试之间存在相互依赖关系时，这可能导致估计偏差。残差相关图和卡方统计量通常用于评估条件独立性假设的有效性，当该假设不成立时，识别哪些测试对是条件依赖的。我们旨在通过一项涵盖广泛场景的模拟研究来评估这些工具的性能。
我们从一个包含四项测试且患病组中测试1和测试2之间存在依赖关系的模型生成数据集。我们改变了样本量、患病率、协方差、敏感性和特异性，总共504种这些因素的组合，每种组合生成1000个数据集。我们在贝叶斯框架下拟合条件独立性模型，并报告绝对偏差、覆盖率，以及残差相关图和统计量在全局或针对每个测试对表明拟合不足的频率。
在所有设置中，残差相关图、成对卡方检验和卡方统计量分别仅在12.1%、10.3%和10.3%的时间内检测到正确的相关测试对，但在64.9%、49.7%和49.5%的时间内错误地表明测试3和测试4之间存在依赖关系。我们观察到在不同参数设置下存在一些差异，当测试3和测试4都比测试1和测试2准确得多时，这些工具似乎更按预期执行。残差相关图、成对卡方检验和卡方统计量分别在74.3%、64.5%和67.5%的模型中识别出整体拟合不足。条件独立性模型倾向于高估相关测试的敏感性（所有场景下的中位数偏差为0.094，第2.5和第97.5百分位数为 -0.003，0.397），并低估患病率和不相关测试的特异性。
不能依靠残差相关图和卡方统计量来识别哪些测试是条件依赖的，并且检测整体拟合不足的能力也相对较低。这很重要，因为未能考虑条件依赖可能导致参数估计出现高度偏差。
