Do Hospital Rankings Mislead Patients? Variability Among National Rating Systems for Orthopaedic Surgery.

INTRODUCTION

A growing number of online hospital rating systems for orthopaedic surgery are found. Although the accuracy and consistency of these systems have been questioned in other fields of medicine, no formal analysis of these systems in orthopaedics has been found.

METHODS

Five hospital rating systems (US News, HealthGrades, CareChex, Women's Choice, and Hospital Compare) were examined which designate "high-performing" and "low-performing" hospitals for orthopaedic surgery. Descriptive analysis was conducted for all hospitals defined as high- or low-performing in any of the five rating systems, and assessment for agreement/disagreement between ratings was done. A subsample of hospitals ranked by all systems was then created, and agreement between rating systems was investigated using a Cohen's kappa. Each hospital was included in a multinomial logistic regression model investigating which hospital characteristics increased the odds of being favorably/unfavorably rated by each system.

RESULTS

One thousand six hundred forty hospitals were evaluated by every rating system. Six hundred thirty-eight unique hospitals were identified as high-performing by at least 1 rating system; however, no hospital was ranked as high-performing by all five rating systems. Four hundred fifty-two unique hospitals were identified as low-performing; however, no hospital was ranked as low-performing by all the three rating systems which define low-performing hospitals. Within the study subsample of hospitals evaluated by each system, little agreement between any combination of rating systems (κ < 0.10) regarding top-tier or bottom-tier performance was found. It was more likely for a hospital to be considered high-performing by one system and low-performing by another (10.66%) than for the majority of the five rating systems to consider a hospital high-performing (3.76%).

CONCLUSION

Little agreement between hospital quality rating systems for orthopaedic surgery is found. Publicly available hospital ratings for performance in orthopaedic surgery offer conflicting results and provide little guidance to patients, providers, or payers when selecting a hospital for orthopaedic surgery.

LEVEL OF EVIDENCE

Level 1 economic study.