Evaluating research training outcomes: experience from the cancer prevention fellowship program at the National Cancer Institute.

PURPOSE

The authors describe an evaluation approach to assess research training that is easy to implement, takes into account individual experience and diversity in research disciplines, and can be adapted to measure various outcomes, depending upon program goals.

METHOD

Using publications as the outcome measure, the authors analyzed data from 66 trainees in the National Cancer Institute's Cancer Prevention Fellowship Program (CPFP) to illustrate this evaluation strategy. For postdoctoral fellows entering the CPFP between 1987 and 1997, the authors considered the three-year period prior to entry in the CPFP (pre-CPFP), the period during training, and the three-year period after completion of the CPFP (post-CPFP). Summary measures for individuals' publications during each of the three time periods were calculated, and the probability of change in total, peer-reviewed, and first-authored publications post-CPFP compared to pre-CPFP was assessed.

RESULTS

Compared to pre-CPFP, the CPFP fellows published significantly more total, peer-reviewed, and first-authored publications post-CPFP. Post-CPFP younger individuals published more than older fellows. MDs had a greater increase in publications over time than did PhDs, but both groups had similar overall numbers of publications post-CPFP. Individuals pursuing a master of public health degree during training published more post-CPFP than did those who did not pursue this training in the program.

CONCLUSIONS

Training programs facing the challenge of evaluating research outcomes will require new evaluation methods that take into account program goals. This easily adaptable, longitudinal evaluation strategy allows for diversity in research disciplines and research experience and can inform programmatic needs and individual progress.