The Hospital for Sick Children, Toronto, ON, Canada.
University of Toronto, Toronto, ON, Canada.
Med Decis Making. 2020 Apr;40(3):314-326. doi: 10.1177/0272989X20912402. Epub 2020 Apr 16.
Investing efficiently in future research to improve policy decisions is an important goal. Expected value of sample information (EVSI) can be used to select the specific design and sample size of a proposed study by assessing the benefit of a range of different studies. Estimating EVSI with the standard nested Monte Carlo algorithm has a notoriously high computational burden, especially when using a complex decision model or when optimizing over study sample sizes and designs. Recently, several more efficient EVSI approximation methods have been developed. However, these approximation methods have not been compared, and therefore their comparative performance across different examples has not been explored. We compared 4 EVSI methods using 3 previously published health economic models. The examples were chosen to represent a range of real-world contexts, including situations with multiple study outcomes, missing data, and data from an observational rather than a randomized study. The computational speed and accuracy of each method were compared. In each example, the approximation methods took minutes or hours to achieve reasonably accurate EVSI estimates, whereas the traditional Monte Carlo method took weeks. Specific methods are particularly suited to problems where we wish to compare multiple proposed sample sizes, when the proposed sample size is large, or when the health economic model is computationally expensive. As all the evaluated methods gave estimates similar to those given by traditional Monte Carlo, we suggest that EVSI can now be efficiently computed with confidence in realistic examples. No systematically superior EVSI computation method exists as the properties of the different methods depend on the underlying health economic model, data generation process, and user expertise.
有效地投资于未来的研究以改善政策决策是一个重要目标。预期样本信息价值（EVSI）可用于通过评估一系列不同研究的效益来选择拟议研究的具体设计和样本量。使用标准嵌套蒙特卡罗算法估计 EVSI 的计算负担非常大，尤其是在使用复杂的决策模型或优化研究样本量和设计时。最近，已经开发了几种更有效的 EVSI 逼近方法。然而，这些近似方法尚未进行比较，因此它们在不同示例中的比较性能尚未得到探索。
我们使用 3 个先前发表的健康经济模型比较了 4 种 EVSI 方法。选择这些示例是为了代表一系列真实情况，包括具有多个研究结果、缺失数据和来自观察性而非随机研究的数据的情况。比较了每种方法的计算速度和准确性。
在每个示例中，近似方法都需要几分钟或几个小时才能得出合理准确的 EVSI 估计值，而传统的蒙特卡罗方法则需要数周时间。特定的方法特别适合于我们希望比较多个建议的样本量的情况，当建议的样本量较大或健康经济模型的计算成本较高时。
由于所有评估的方法都给出了与传统蒙特卡罗相似的估计值，因此我们建议现在可以在现实示例中以有信心的方式高效地计算 EVSI。由于不同方法的性质取决于基础健康经济模型、数据生成过程和用户专业知识，因此不存在系统上优越的 EVSI 计算方法。
