Can online networks provide quality answers to questions about occupational safety and health?

OBJECTIVES

To assess whether experts can provide high-quality answers to occupational safety and health (OSH) questions in online Question & Answer (Q&A) networks.

METHODS

The authors evaluated the quality of answers provided by qualified experts in two Dutch online networks: ArboAntwoord and the Helpdesk of the Netherlands Center for Occupational Diseases. A random sample of 594 answers was independently evaluated by two raters using nine answer quality criteria. An additional criterion, the agreement of answers with the best available evidence, was explored by peer review of a sample of 42 answers. Reviewers performed an evidence search in Medline.

RESULTS

The median answer quality score of ArboAntwoord (N=295) and the Netherlands Center for Occupational Diseases Helpdesk (N=299) was 8 of 9 (IQR 2). The inter-rater reliability of the first nine quality criteria was high (κ 0.82-0.90, p<0.05). A question answered by two or more experts had a greater probability of a high-quality score than questions answered by one expert (OR 4.9, 95% CI 2.7 to 9.0). Answers most often scored insufficient on the use of evidence to underpin the answer (36% and 38% for the networks, respectively) and on conciseness (35% and 31%, respectively). Peer review demonstrated that 43%-72% of the answers in both online networks were in complete agreement with the best available evidence.

CONCLUSIONS

OSH experts are able to provide quality answers in online OSH Q&A networks. Our answer quality appraisal instrument was feasible and provided information on how to improve answer quality.