Quality of out-of-hours telephone triage by general practitioners and nurses: development and testing of the AQTT - an assessment tool measuring communication, patient safety and efficiency.

OBJECTIVE

To develop a valid and reliable assessment tool able to measure quality of communication, patient safety and efficiency in out-of-hours (OOH) telephone triage conducted by both general practitioners (GP) and nurses.

DESIGN

The Dutch KERNset tool was translated into Danish and supplemented with items from other existing tools. Face validity, content validity and applicability in OOH telephone triage (OOH-TT) were secured through a two-round Delphi process involving relevant stakeholders. Forty-eight OOH patient contacts were assessed by 24 assessors in test-retest and inter-rater designs.

SETTING

OOH-TT services in Denmark conducted by GPs, nurses or doctors with varying medical specialisation.

PATIENTS

Audio-recorded OOH patient contacts.

MAIN OUTCOME MEASURES

Test-retest and inter-rater reliability were analysed using ICC, Fleiss' kappa and percent agreement.

RESULTS

Major adaptations during the Delphi process were made. The 24-item assessment tool (Assessment of Quality in Telephone Triage - AQTT) measured communicative quality, health-related quality and four overall quality aspects. The test-retest ICC reliability was good for the overall quality of communication (0.85), health-related quality (0.83), patient safety (0.81) and efficiency (0.77) and satisfactory when assessing specific aspects. Inter-rater reliability revealed reduced reliability in ICC and in Fleiss' kappa. Percent agreement revealed satisfactory agreements when differentiating between 'poor' and 'sufficient' quality).

CONCLUSION

The AQTT demonstrated high face, content and construct validity, satisfactory test-retest reliability, reduced inter-rater reliability, but satisfactory percent agreement when differentiating between 'poor' and 'sufficient' quality. The AQTT was found feasible and clinically relevant for assessing the quality of GP- and nurse-led OOH-TT. KEYPOINTS Comparative knowledge is sparse regarding quality of out-of-hours telephone triage conducted by general practitioners and nurses. The assessment tool (AQTT) enables assessment of quality in OOH telephone triage conducted by nurses and general practitioners AQTT is feasible and clinically relevant for assessment of communication, patient safety and efficiency. AQTT can be used to identify areas for improvement in telephone triage.