García-Nieto Rebeca, Parra Uribe Isabel, Palao Diego, Lopez-Castroman Jorge, Sáiz Pilar Alejandra, García-Portilla María Paz, Saiz Ruiz Jerónimo, Ibañez Angela, Tiana Thais, Durán Sindreu Santiago, Perez Sola Victor, de Diego-Otero Yolanda, Pérez-Costillas Lucia, Fernández García-Andrade Rafael, Saiz-González Dolores, Jiménez Arriero Miguel Angel, Navío Acosta Mercedes, Giner Lucas, Guija Julio Antonio, Escobar José Luis, Cervilla Jorge Antonio, Quesada Marta, Braquehais Dolores, Blasco-Fontecilla Hilario, Legido-Gil Teresa, Aroca Fuensanta, Baca-García Enrique
Fundación Jiménez Díaz, Madrid, CIBERSAM, España.
Rev Psiquiatr Salud Ment. 2012 Jan-Mar;5(1):24-36. doi: 10.1016/j.rpsm.2011.10.001. Epub 2011 Nov 16.
Inter-rater agreement is a crucial aspect in the planning and performance of a clinical trial in which the main assessment tool is the clinical interview. The main objectives of this study are to study the inter-rater agreement of a tool for the assessment of suicidal behavior (Brief Suicide Questionnaire) and to examine whether the inter-examiner agreement when multiple ratings are made on a single subject is an efficient method to assess the reliability of an instrument.
In the context of designing a multicenter clinical trial, 32 psychiatrists assessed a videotaped clinical interview of a patient with suicidal behavior. In order to identify those items in which a greater level of discordance existed and detect the examiners whose ratings differed significantly from the average ratings, we used the DOMENIC method (Detecion of Multiple Examiners Not in Consensus).
Inter-rater agreement was between poor (<70%) to excelent (90-100%. Inter-rater agreement in Brugha's list of threatening experiences ranged from 75.5 and 100%; in the Global Assessment of Functioning (GAF) Scale was 82.58%; in the Beck's Suicidal Intent Scale, ranged from 67.5 and 97%; in the Beck's Scale for Suicide Ideation, ranged from 63.5 and 100%; and in the Lethality Rating Scale was 88.39%. On the whole, the level of agreement among raters, both in general scores and in particular items, was appropriate.
The proposed design allows the assessment of the inter-rater agreement in an efficient way (only in one session). In addition, regarding the Brief Suicide Questionnaire, inter-raters agreement was appropriate.
