Effect of XML markup on retrieval of clinical documents.

OBJECTIVE

To determine the effect on clinical information retrieval of structuring typical clinical documents in XML, according to the general guidelines of Health Level Seven's Clinical Document Architecture.

METHODS

One thousand clinical documents of eight frequently occurring types were deidentified and marked up in XML for access using a Web browser. Fifty information-seeking tasks were posed to subjects. The tasks were comprised of two typical clinical question types-individual patient results reporting and cohort identification. A control group of physician subjects could perform only free-text, keyword searching. The treatment group's interface permitted field-based searching of particular sections within each document. Differences in precision and other measures of search success across and between question types were investigated for statistical significance.

RESULTS

No statistically significant differences were found between the control and treatment conditions in mean time elapsed or the mean number of records in the final result set. In fact, tasks performed in the treatment condition required a mean number of more steps in the search sequence to a degree that was statistically significant. Tasks performed in the treatment condition had a statistically significant lower rate of mean precision. There was no statistically significant difference between the means of relevance of the individual patient and cohort identification tasks.

CONCLUSION

These findings are in line with Tange et al. who found that coarser granularity of clinical narrative gave better results. The results of this experiment also have implications for automatic text processing. Complex tag sets cannot ultimately resolve problems of unstandardized structure; the lack of existing structure within clinical documents is itself a significant limitation.