1 Partnership for Health Analytic Research, Beverly Hills, California.
2 Tufts Medical Center, Boston, Massachusetts.
J Manag Care Spec Pharm. 2017 Jun;23(6-a Suppl):S34-S48. doi: 10.18553/jmcp.2017.23.6-a.s34.
Several organizations have developed frameworks to systematically assess the value of new drugs.
To evaluate the convergent validity and interrater reliability of 4 value frameworks to understand the extent to which these tools can facilitate value-based treatment decisions in oncology.
Eight panelists used the American Society of Clinical Oncology (ASCO), European Society for Medical Oncology (ESMO), Institute for Clinical and Economic Review (ICER), and National Comprehensive Cancer Network (NCCN) frameworks to conduct value assessments of 15 drugs for advanced lung and breast cancers and castration-refractory prostate cancer. Panelists received instructions and published clinical data required to complete the assessments, assigning each drug a numeric or letter score. Kendall's Coefficient of Concordance for Ranks (Kendall's W) was used to measure convergent validity by cancer type among the 4 frameworks. Intraclass correlation coefficients (ICCs) were used to measure interrater reliability for each framework across cancers. Panelists were surveyed on their experiences.
Kendall's W across all 4 frameworks for breast, lung, and prostate cancer drugs was 0.560 (P= 0.010), 0.562 (P = 0.010), and 0.920 (P < 0.001), respectively. Pairwise, Kendall's W for breast cancer drugs was highest for ESMO-ICER and ICER-NCCN (W = 0.950, P = 0.019 for both pairs) and lowest for ASCO-NCCN (W = 0.300, P = 0.748). For lung cancer drugs, W was highest pairwise for ESMO-ICER (W = 0.974, P = 0.007) and lowest for ASCO-NCCN (W = 0.218, P = 0.839); for prostate cancer drugs, pairwise W was highest for ICER-NCCN (W = 1.000, P < 0.001) and lowest for ESMO-ICER and ESMO-NCCN (W = 0.900, P = 0.052 for both pairs). When ranking drugs on distinct framework subdomains, Kendall's W among breast cancer drugs was highest for certainty (ICER, NCCN: W = 0.908, P = 0.046) and lowest for clinical benefit (ASCO, ESMO, NCCN: W = 0.345, P = 0.436). Among lung cancer drugs, W was highest for toxicity (ASCO, ESMO, NCCN: W = 0. 944, P < 0.001) and lowest for certainty (ICER, NCCN: W = 0.230, P = 0.827); and among prostate cancer drugs, it was highest for quality of life (ASCO, ESMO: W = 0.986, P = 0.003) and lowest for toxicity (ASCO, ESMO, NCCN: W = 0.200, P = 0.711). ICC (95% CI) for ASCO, ESMO, ICER, and NCCN were 0.800 (0.660-0.913), 0.818 (0.686-0.921), 0.652 (0.466-0.834), and 0.153 (0.045-0.371), respectively. When scores were rescaled to 0-100, NCCN provided the narrowest band of scores. When asked about their experiences using the ASCO, ESMO, ICER, and NCCN frameworks, panelists generally agreed that the frameworks were logically organized and reasonably easy to use, with NCCN rated somewhat easier.
Convergent validity among the ASCO, ESMO, ICER, and NCCN frameworks was fair to excellent, increasing with clinical benefit subdomain concordance and simplicity of drug trial data. Interrater reliability, highest for ASCO and ESMO, improved with clarity of instructions and specificity of score definitions. Continued use, analyses, and refinements of these frameworks will bring us closer to the ultimate goal of using value-based treatment decisions to improve patient care and outcomes.
This work was funded by Eisai Inc. Copher and Knoth are employees of Eisai Inc. Bentley, Lee, Zambrano, and Broder are employees of Partnership for Health Analytic Research, a health services research company paid by Eisai Inc. to conduct this research. For this study, Cohen, Huynh, and Neville report fees from Partnership for Health Analytic Research. Outside of this study, Cohen receives grants and direct consulting fees from various companies that manufacture and market pharmaceuticals. Mei reports a grant from Eisai Inc. during this study. The other authors have no disclosures to report. Study concept and design were contributed by Bentley and Broder, with assistance from Elkin and Cohen. Bentley took the lead in data collection, along with Elkin, Huynh, Mukherjea, Neville, Mei, Popescu, Lee, and Zambrano. Data interpretation was performed by Bentley and Broder, along with Elkin, Cohen, Copher, and Knoth. The manuscript was written primarily by Bentley, along with Elkin and Broder, and revised by Bentley, Broder, Elkin, Cohen, Copher, and Knoth. Select components of this work's methods were presented at ISPOR 19th Annual European Congress held in Vienna, Austria, October 29-November 2, 2016, and Society for Medical Decision Making 38th Annual North American Meeting held in Vancouver, Canada, October 23-26, 2016.
有几个组织已经制定了框架，以系统地评估新药的价值。
评估 4 种价值框架的收敛有效性和组内一致性，以了解这些工具在肿瘤学中促进基于价值的治疗决策的程度。
8 名小组成员使用美国临床肿瘤学会（ASCO）、欧洲肿瘤内科学会（ESMO）、临床与经济审查研究所（ICER）和国家综合癌症网络（NCCN）框架对 15 种用于治疗晚期肺癌和乳腺癌以及去势抵抗性前列腺癌的药物进行价值评估。小组成员收到了完成评估所需的说明和已发表的临床数据，对每种药物进行了数字或字母评分。采用 Kendall 等级相关系数（Kendall's W）衡量 4 种框架中癌症类型的收敛有效性。使用组内相关系数（ICC）衡量每个框架在不同癌症类型之间的组内一致性。对小组成员的经验进行了调查。
所有 4 种框架在乳腺癌、肺癌和前列腺癌药物方面的 Kendall's W 分别为 0.560（P=0.010）、0.562（P=0.010）和 0.920（P<0.001）。两两比较时，乳腺癌药物中 ESMO-ICER 和 ICER-NCCN 之间的 Kendall's W 最高（W=0.950，P=0.019），而 ASCO-NCCN 之间的 Kendall's W 最低（W=0.300，P=0.748）。在肺癌药物方面，ESMO-ICER 之间的 Kendall's W 最高（W=0.974，P=0.007），ASCO-NCCN 之间的 Kendall's W 最低（W=0.218，P=0.839）；在前列腺癌药物方面，ICER-NCCN 之间的 Kendall's W 最高（W=1.000，P<0.001），ESMO-ICER 和 ESMO-NCCN 之间的 Kendall's W 最低（W=0.900，P=0.052）。在对不同框架子域的药物进行排名时，乳腺癌药物中，Kendall's W 对确定性（ICER、NCCN：W=0.908，P=0.046）的一致性最高，对临床获益（ASCO、ESMO、NCCN：W=0.345，P=0.436）的一致性最低。在肺癌药物中，毒性（ASCO、ESMO、NCCN：W=0.944，P<0.001）的 Kendall's W 最高，确定性（ICER、NCCN：W=0.230，P=0.827）的 Kendall's W 最低；在前列腺癌药物中，Kendall's W 对生活质量（ASCO、ESMO：W=0.986，P=0.003）的一致性最高，对毒性（ASCO、ESMO、NCCN：W=0.200，P=0.711）的一致性最低。ASCO、ESMO、ICER 和 NCCN 的 ICC（95%CI）分别为 0.800（0.660-0.913）、0.818（0.686-0.921）、0.652（0.466-0.834）和 0.153（0.045-0.371）。当分数被重新调整为 0-100 时，NCCN 提供了最窄的分数范围。当被问及使用 ASCO、ESMO、ICER 和 NCCN 框架的经验时，小组成员普遍认为这些框架逻辑清晰，使用合理，其中 NCCN 被认为稍微简单一些。
ASCO、ESMO、ICER 和 NCCN 框架之间的收敛有效性为中等至良好，随着临床获益子域的一致性和药物试验数据的简化而增加。组内一致性最高的是 ASCO 和 ESMO，其提高的原因是说明更加清晰，评分定义更加具体。这些框架的持续使用、分析和改进将使我们更接近使用基于价值的治疗决策来改善患者护理和结果的最终目标。
这项工作得到了 Eisai Inc. 的资助。Copher 和 Knoth 是 Eisai Inc. 的员工。Bentley、Lee、Zambrano 和 Broder 是 Partnership for Health Analytic Research 的员工，这是一家为 Eisai Inc. 进行这项研究的卫生服务研究公司。就这项研究而言，Cohen、Huynh 和 Neville 报告了来自 Partnership for Health Analytic Research 的费用。除此之外，Cohen 还因制造和销售药品的各种公司获得了直接咨询费和研究费。Mei 报告说在研究期间收到了 Eisai Inc. 的赠款。其他作者没有披露信息。研究的概念和设计由 Bentley 和 Broder 提出，并得到了 Elkin 和 Cohen 的协助。Bentley 带头进行了数据收集，同时还有 Elkin、Huynh、Mukherjea、Neville、Mei、Popescu、Lee 和 Zambrano。数据解释由 Bentley 和 Broder 与 Elkin、Cohen、Copher 和 Knoth 共同完成。手稿主要由 Bentley 撰写，同时还有 Elkin 和 Broder，修订由 Bentley、Broder、Elkin、Cohen、Copher 和 Knoth 完成。这项工作的部分方法在 2016 年 10 月 29 日至 11 月 2 日在奥地利维也纳举行的第 19 届 ISPOR 欧洲大会和 2016 年 10 月 23 日至 26 日在加拿大温哥华举行的第 38 届社会医疗决策学会北美会议上进行了介绍。
