Interobserver agreement in the interpretation of single-field digital fundus images for diabetic retinopathy screening.

PURPOSE

To assess agreement among a group of ophthalmic care providers, including ophthalmologists and trained nonphysician personnel, in the interpretation of single-field digital fundus images for diabetic retinopathy screening.

DESIGN

Interobserver reliability study.

PARTICIPANTS

Twelve ophthalmic care personnel, including 3 retina specialists, 3 general ophthalmologists, 3 ophthalmic nurses, and 3 ophthalmic photographers.

METHODS

All participants were to read 400 good single-field digital fundus images of diabetic patients from a community hospital. The nonphysician personnel group read the images 1 month after attending a 2-day intensive instruction course regarding diabetic retinopathy screening. The ophthalmologists read the images without additional training. The 3 retina specialists read the images again together 2 months later to form a consensus regarding retinopathy severity and macular edema for each case. All readers used the Early Treatment Diabetic Retinopathy Study standard photographs as guidelines.

MAIN OUTCOME MEASURES

The kappa statistic was used for the reliability assessment of the diabetic retinopathy severity and macular edema, and for the identification of cases that needed referral to ophthalmologists.

RESULTS

There is only fair agreement among all readers. The multirater kappa coefficient for retinopathy severity is 0.34; for macular edema, 0.27; and for referral cases, 0.28. Retina specialists have the best agreement among all groups (kappa = 0.58 for retinopathy severity or macular edema, kappa = 0.63 for referrals). There is also fair agreement when all readers are compared with the consensus of retina specialists (kappas = 0.35, 0.28, and 0.29 for retinopathy severity, macular edema, and referrals, respectively), and the retina specialist group also has the best agreement (kappas = 0.63, 0.65, and 0.67 for retinopathy severity, macular edema, and referrals).

CONCLUSIONS

Without additional training, retina specialists may be the most reliable personnel to interpret single-field digital fundus images for diabetic retinopathy screening. For other ophthalmic care personnel to achieve comparable reliability, a comprehensive instruction course with specific continuing education is essential. Authorized nonphysician interpreters should be experts, and new standard photographs for single-field digital fundus image interpretation may also be required to improve interobserver reliability.