Wood Fred B, Siegel Elliot R, Feldman Sue, Love Cynthia B, Rodrigues Dennis, Malamud Mark, Lagana Marie, Crafts Jennifer
National Library of Medicine, National Institutes of Health, US Department of Health and Human Services, Bethesda, MD 20894, USA.
J Med Internet Res. 2008 Feb 15;10(1):e4. doi: 10.2196/jmir.944.
The National Institutes of Health (NIH), US Department of Health and Human Services (HHS), realized the need to better understand its Web users in order to help assure that websites are user friendly and well designed for effective information dissemination. A trans-NIH group proposed a trans-NIH project to implement an online customer survey, known as the American Customer Satisfaction Index (ACSI) survey, on a large number of NIH websites-the first "enterprise-wide" ACSI application, and probably the largest enterprise Web evaluation of any kind, in the US government. The proposal was funded by the NIH Evaluation Set-Aside Program for two years at a cost of US $1.5 million (US $1.275 million for survey licenses for 60 websites at US $18000 per website; US $225,000 for a project evaluation contractor).
The overall project objectives were to assess the value added to the participating NIH websites of using the ACSI online survey, identify any NIH-wide benefits (and limitations) of the ACSI, ascertain any new understanding about the NIH Web presence based on ACSI survey results, and evaluate the effectiveness of a trans-NIH approach to Web evaluation. This was not an experimental study and was not intended to evaluate the ACSI survey methodology, per se, or the impacts of its use on customer satisfaction with NIH websites.
The evaluation methodology included baseline pre-project websites profiles; before and after email surveys of participating website teams; interviews with a representative cross-section of website staff; observations of debriefing meetings with website teams; observations at quarterly trans-NIH Web staff meetings and biweekly trans-NIH leadership team meetings; and review and analysis of secondary data.
Of the original 60 NIH websites signed up, 55 implemented the ACSI survey, 42 generated sufficient data for formal reporting of survey results for their sites, and 51 completed the final project survey. A broad cross-section of websites participated, and a majority reported significant benefits and new knowledge gained from the ACSI survey results. NIH websites as a group scored consistently higher on overall customer satisfaction relative to US government-wide and private sector benchmarks.
Overall, the enterprise-wide experiment was successful. On the level of individual websites, the project confirmed the value of online customer surveys as a Web evaluation method. The evaluation results indicated that successful use of the ACSI, whether site-by-site or enterprise-wide, depends in large part on strong staff and management support and adequate funding and time for the use of such evaluative methods. In the age of Web-based e-government, a broad commitment to Web evaluation may well be needed. This commitment would help assure that the potential of the Web and other information technologies to improve customer and citizen satisfaction is fully realized.
美国卫生与公众服务部（HHS）下属的国立卫生研究院（NIH）意识到，有必要更好地了解其网站用户，以确保网站对用户友好且设计合理，便于有效传播信息。一个跨NIH的团队提出了一个跨NIH项目，即在大量NIH网站上开展一项在线客户满意度调查，即美国客户满意度指数（ACSI）调查，这是美国政府首个“全机构范围”的ACSI应用，可能也是规模最大的全机构范围的任何类型的网络评估。该提案由NIH评估预留计划资助，为期两年，费用为150万美元（其中127.5万美元用于60个网站的调查许可，每个网站18000美元；22.5万美元用于项目评估承包商）。
项目的总体目标是评估使用ACSI在线调查为参与的NIH网站带来的附加值，确定ACSI在全NIH范围内的益处（和局限性），根据ACSI调查结果确定对NIH网站形象的新认识，并评估跨NIH网络评估方法的有效性。这不是一项实验性研究，也并非旨在评估ACSI调查方法本身，或其使用对用户对NIH网站满意度的影响。
评估方法包括项目前网站的基线概况；对参与网站团队进行电子邮件调查前后的情况；与具有代表性的网站工作人员进行访谈；观察与网站团队的汇报会议；观察每季度的跨NIH网络工作人员会议和每两周一次的跨NIH领导团队会议；以及对二手数据的审查和分析。
最初报名的60个NIH网站中，55个实施了ACSI调查，42个生成了足够的数据用于正式报告其网站的调查结果，51个完成了最终项目调查。众多不同类型的网站参与其中，大多数网站报告称从ACSI调查结果中获得了显著益处和新知识。相对于美国政府整体和私营部门的基准，NIH网站作为一个整体在总体客户满意度方面得分一直较高。
总体而言，全机构范围的实验是成功的。在单个网站层面，该项目证实了在线客户调查作为一种网络评估方法的价值。评估结果表明，成功使用ACSI，无论是逐站点还是全机构范围，在很大程度上取决于强大的工作人员和管理支持，以及使用此类评估方法的充足资金和时间。在基于网络的电子政务时代，可能非常需要对网络评估做出广泛承诺。这一承诺将有助于确保充分实现网络和其他信息技术在提高客户和公民满意度方面的潜力。
