Evaluation of health alerts from an early illness warning system in independent living.

Passive sensor networks were deployed in independent living apartments to monitor older adults in their home environments to detect signs of impending illness and alert clinicians so they can intervene and prevent or delay significant changes in health or functional status. A retrospective qualitative deductive content analysis was undertaken to refine health alerts to improve clinical relevance to clinicians as they use alerts in their normal workflow of routine care delivery to older adults. Clinicians completed written free-text boxes to describe actions taken (or not) as a result of each alert; they also rated the clinical significance (relevance) of each health alert on a scale of 1 to 5. Two samples of the clinician's written responses to the health alerts were analyzed after alert algorithms had been adjusted based on results of a pilot study using health alerts to enhance clinical decision-making. In the first sample, a total of 663 comments were generated by seven clinicians in response to 385 unique alerts; there are more comments than alerts because more than one clinician rated the same alert. The second sample had a total of 142 comments produced by three clinicians in response to 88 distinct alerts. The overall clinical relevance of the alerts, as judged by the content of the qualitative comments by clinicians for each alert, improved from 33.3% of the alerts in the first sample classified as clinically relevant to 43.2% in the second. The goal is to produce clinically relevant alerts that clinicians find useful in daily practice. The evaluation methods used are described to assist others as they consider building and iteratively refining health alerts to enhance clinical decision making.