Fellow of the American Dietetic Association credentialing program: development and implementation of a portfolio-based assessment.

This report has described the measurement and operational procedures used for the design, development, and implementation of the newly inaugurated Fellow of The American Dietetic Association credentialing program. The program was established to identify and certify registered dietitians who possess the characteristics of advanced-level practice. The main features of the Fellow program are as follows. It is based on the results of an empirical study that identified the characteristics of advanced-level practice. It is guided by the Fellow Assessment Plan, which specifies the content and level of dietetics practice involved and the psychometric and operational requirements for sound measurement. It is a performance-based portfolio assessment of six professional characteristics (education, work experience, professional achievement, professional roles, professional contacts, and approach to practice). It uses objective automated computer scoring and independent peer-review grading of candidate materials in relation to an explicit scoring rubric. It requires statistical analysis of the database of candidate scores, which is conducted to ensure that the scoring of candidate materials is accurate and reliable and that the assessment, as a measurement instrument, is psychometrically sound. It involves a standard-setting workshop in which a panel of subject-matter experts establish the lowest level of acceptable candidate performance on Approach to Practice; the passing standard for the other five characteristics was preestablished by the empirical study. A careful review of all evidence on the development and implementation of the 1994 Fellow assessment suggests that these procedures are technically sound and produce psychometrically defensible results. Our evaluation and commentary from CDR and CTB staff, subject-matter expert participants, and the candidates themselves suggested a number of ways to improve the Fellow program in subsequent administrations of the assessment. Changes have been implemented and CDR will continue to consider all data in its ongoing improvement efforts.