PURPOSE

The study tested the inter-rater reliability and accuracy of triage nurses' assignment of urgency ratings for mental health patient scenarios based on the 2008 Canadian Triage and Acuity Scale (CTAS) guidelines, using a standardized triage tool. The influence of triage experience, educational preparation, and comfort level with mental health presentations on the accuracy of urgency ratings was also explored.

METHODS

Study participants assigned urgency ratings to 20 mental health patient scenarios in randomized order using the CTAS. The scenarios were developed using actual triage notes and were reviewed by an expert panel of emergency and mental health clinicians for face and content validity.

RESULTS

The overall Fleiss' kappa, the measure of inter-rater reliability for this sample of triage nurses (n=18), was 0.312, representing only fair albeit statistically significant (P<0.0001) agreement. Kendall's coefficient of concordance for the sample was calculated to be 0.680 (P<0.0001), which signifies moderate agreement. Although the sample reported high levels of education, comfort with mental health presentations, and experience, accuracy in urgency ratings measured by the percentage of correct responses ranged from 0.05% to 94% (mean: 54%). Greater accuracy in urgency ratings was recorded for triage nurses who used second-order modifiers and avoided the use of override.

CONCLUSION

Specific focus on the use of second-order modifiers in orientation and ongoing education of triage nurses may improve the reliability and validity of the CTAS when used to assign urgency ratings to mental health presentations.