PURPOSE OF THE STUDY The current study aimed to investigate the intra- and inter-observer reliability of the Ellman classification system in partialthickness rotator cuff tears through magnetic resonance imaging (MRI) scans instead of arthroscopic views. MATERIAL AND METHODS Pre-operative MRI scans of 45 patients, with confirmed partial-thickness rotator cuff rupture in previous arthroscopic surgeries (performed by the senior author), were obtained from Picture Archiving and Communication Systems records. The observers (n=8) were asked to categorize MRI scans according to Ellman's classification of location and grade. There were four orthopedic surgeons less experienced in rotator cuff operations in the first group and four more experienced orthopedic surgeons in the second group. They were asked to re-evaluate the MRI scans six weeks later, without access to their previous answers. Reliability evaluation was performed within and among the groups. It was also evaluated if the surgeon's experience increased the reliability of the classification. Fleiss kappa coefficient was used for the inter-observer reliability and Cohen kappa coefficient for the intra-observer reliability, and post hoc analysis was performed. RESULTS When all observers were examined in the inter-observer evaluation, it was seen that there was moderate agreement in the first location evaluation (κ=0.414); however, there was fair agreement in all other evaluations in both groups (κ=0.339- 0.383-0.337, respectively). When all observers were examined in the mean intra-observer evaluation, it was seen that there was substantial agreement in both evaluations (κ=0.795-0.721, respectively). DISCUSSION A classification system must be valid, reliable, and reproducible. It should establish a standard terminology for both surgeons and researchers. The correct identification of the tear configuration is crucial for selecting the correct repair technique. In our study, in which we evaluated the Ellman classification, which is frequently used in arthroscopic diagnosis, we investigated its intra-observer and inter-observer reliability on MRI scans. Although the mean intra-observer evaluation results were substantial agreement (κ=0.795-0.721, respectively), inter-observer evaluation results were fair agreement (κ=0.339- 0.383-0.337, respectively) except for the first location evaluation (κ=0.414). CONCLUSIONS Although intra-observer reliability was satisfactory, the Ellman system used in the classification of partial-thickness rotator cuff tears was not found to be useful by using only MRI views because of fair inter-observer reliability except for the first location evaluation, which was moderate agreement. Key words: partial, rotator cuff, tear, Ellman classification, reliability, validity.