Patient and Caregiver Engagement, Memorial Sloan Kettering Cancer Center, New York, USA.
Patient and Caregiver Engagement, Memorial Sloan Kettering Cancer Center, New York, USA.
Patient Educ Couns. 2018 Mar;101(3):490-496. doi: 10.1016/j.pec.2017.09.003. Epub 2017 Sep 6.
To assess the interrater reliability (IRR) and usability of the Patient Education Materials Assessment Tool (PEMAT) and the relationship between PEMAT scores and readability levels.
One hundred ten materials (80 print, 30 audiovisual) were evaluated, each by two raters, using the PEMAT. IRR was calculated using Gwet's AC1 and summarized across items in each PEMAT domain (understandability and actionability) and by material type. A survey was conducted to solicit raters' experience using the PEMAT. Readability of each material was assessed using the SMOG Index.
The median IRR was 0.92 for understandability and 0.93 for actionability across all relevant items, indicating good IRR. Eight PEMAT items had Gwet's AC1 values less than 0.81. PEMAT and SMOG Index scores were inversely correlated, with a Spearman's rho of -0.20 (p=0.081) for understandability and -0.15 (p=0.194) for actionability. While 92% of raters agreed the PEMAT was easy to use, survey results suggested specific items for clarification.
While the PEMAT demonstrates moderate to excellent IRR overall, amendments to items with lower IRR may increase the usefulness of the tool.
The PEMAT is a useful supplement to reading level alone in the assessment of educational materials.
评估患者教育材料评估工具（PEMAT）的评分者间信度（IRR）和可用性，以及 PEMAT 评分与可读性水平之间的关系。
使用 PEMAT 对 110 种材料（80 种印刷品，30 种视听材料）进行评估，每种材料由两名评分者进行评估。使用 Gwet 的 AC1 计算 IRR，并汇总 PEMAT 每个领域（可理解性和可操作性）和每种材料类型的所有项目的 IRR。进行了一项调查，以征求评分者使用 PEMAT 的经验。使用简单平均五级评分法（SMOG）指数评估每种材料的可读性。
所有相关项目的可理解性和可操作性的中位数 IRR 分别为 0.92 和 0.93，表明 IRR 良好。八项 PEMAT 项目的 Gwet 的 AC1 值低于 0.81。PEMAT 和 SMOG 指数得分呈负相关，可理解性的 Spearman rho 为-0.20（p=0.081），可操作性的 Spearman rho 为-0.15（p=0.194）。虽然 92%的评分者认为 PEMAT 易于使用，但调查结果表明需要对特定项目进行澄清。
虽然 PEMAT 总体上显示出中等至良好的 IRR，但对 IRR 较低的项目进行修订可能会提高该工具的有用性。
在评估教育材料时，PEMAT 是阅读水平的有用补充。
