Improvement in precision of counting actinic keratoses.

BACKGROUND

Actinic keratoses (AKs) often serve as a primary endpoint for clinical studies. However, reliability of counting these lesions is poor, even among expert dermatologists.

OBJECTIVES

To investigate the reliability of counting AKs before and after a yearly consensus meeting, held annually for 4 years.

METHODS

As part of the Veterans Affairs (VA) Keratinocyte Carcinoma Chemoprevention Trial, board-certified dermatologists convened annually for 4 years to individually count the number of actinic keratoses on three to five test subjects. The dermatologists then met as a group for a consensus discussion on what constituted an AK lesion on each subject. Afterwards, each dermatologist repeated the independent counting exercise on three to five new subjects. The intraclass correlation coefficient (ICC) was used to analyze the reliability of counting AKs among the dermatologists.

RESULTS

Eight dermatologists participated in this exercise for 4 consecutive years. Pre-consensus discussion ICCs over 4 years were 0.18, 0.34, 0.38, 0.75, respectively, showing sustained improvement with each consensus discussion. The greatest improvement in reliability of AK counts was shown during the first year of consensus discussions, when the ICC improved from 0.18 to 0.67. There was no improvement by the fourth year of consensus discussion, with pre- and post-consensus ICCs of 0.75 and 0.75, respectively.

CONCLUSIONS

Annual consensus discussions can lead to improvement in reliability of AK counts. This improvement was sustained over 4 years. By the fourth year, the discussion meeting had no effect on improvement in reliability. A consensus meeting discussion may be helpful for improving reliability in other trials.