Preliminary Evaluation of a Guideline Classification System

Authors

Elmer Bernstam, Stanford Medical Informatics, Stanford University School of Medicine, Stanford, CA
Nachman Ash, 2Decision Systems Group, Harvard Medical School, Brigham & Women's Hospital, Boston, MA
Mor Peleg, 'Stanford Medical Informatics, Stanford University School of Medicine, Stanford, CA
Samson Tu, 'Stanford Medical Informatics, Stanford University School of Medicine, Stanford, CA
Edward H. Shortliffe, 3Department of Medical Informatics, Columbia University, New York, NY
Robert A. Greenes, 2Decision Systems Group, Harvard Medical School, Brigham & Women's Hospital, Boston, MA

Publication Date

2001

Journal

Proc AMIA Symp. 2001; 863.

Abstract

In a previous paper, we presented a proposed expansion of the National Guideline Clearing-house (NGC) classification1. We performed a preliminary evaluation of the classification based on 100 guidelines randomly selected from the NGC collection. We found that 89 of the 100 guidelines could be assigned to a single guideline category. To test inter-observer agreement, twenty guidelines were also categorized by a second investigator. Agreement was found to be 40-90% depending on the axis, which compares favorably with agreement among MeSH indexers (30-60%)2. We conclude that categorization is feasible. Further research is needed to clarify axes with poor inter-observer agreement.

Comments

PMCID: PMC2243672