Evidence-based clinical decision support.

BACKGROUND: Clinical decision support (CDS) is a key tool for enabling evidence-based medicine and improving the quality of healthcare. However, effective CDS faces a variety of challenges, including those relating to knowledge synthesis, capture, transformation, localization and maintenance. If not properly addressed, these challenges can limit the effectiveness of CDS, and potentially risk inaccurate or inappropriate interventions to clinicians. OBJECTIVES: (1) To describe an approach to CDS development using evidence as a basis for clinical decision support systems that promote effective care; (2) To review recent evidence regarding the effectiveness of selected clinical decision support systems. METHOD: Review and analysis of recent literature with identification of trends and best practices. RESULTS: The state-of-the-art in CDS has advanced significantly, and many recent trials have shown CDS to be effective, although the results are mixed overall. Issues related to knowledge capture and synthesis, problems in knowledge transformation at the interface between knowledge authors and CDS developers, and problems specific to local CDS design and implementation can interfere with CDS development. Best practices, tools and techniques to manage them are described. CONCLUSIONS: CDS, when used well, can be effective, but further research is needed for it to reach its full potential.

How "should" we write guideline recommendations? Interpretation of deontic terminology in clinical practice guidelines: survey of the health services community.

OBJECTIVE: To describe the level of obligation conveyed by deontic terms (words such as "should", "may", "must" and "is indicated") commonly found in clinical practice guidelines. DESIGN: Cross-sectional electronic survey. SETTING: A clinical scenario was developed by the researchers, and recommendations containing 12 deontic terms and phrases were presented to the participants. PARTICIPANTS: All 1332 registrants of the 2008 annual conference of the US Agency for Healthcare Research and Quality. MAIN OUTCOME MEASURES: Participants indicated the level of obligation they believed guideline authors intended by using a slider mechanism ranging from "No obligation" (leftmost position recorded as 0) to "Full obligation" (rightmost position recorded as 100.) RESULTS: 445/1332 registrants (36%) submitted the on-line survey; 254/445 (57%) reported that they have experience in developing clinical practice guidelines; 133/445 (30%) indicated that they provide healthcare. "Must" conveyed the highest level of obligation (median = 100) and least amount of variability (interquartile range = 5.) "May" (median = 37) and "may consider" (median = 33) conveyed the lowest levels of obligation. All other terms conveyed intermediate levels of obligation characterised by wide and overlapping interquartile ranges. CONCLUSIONS: Members of the health services community believe guideline authors intend variable levels of obligation when using different deontic terms within practice recommendations. Ranking of a subset of terms by intended level of obligation is possible. Matching deontic terminology to the intended recommendation strength can help standardise the use of deontic terminology by guideline developers.

Information technology for children's health and health care: report on the Information Technology in Children's Health Care Expert Meeting, September 21-22, 2000.

In September 2000, the Agency for Healthcare Quality and Research and the American Academy of Pediatrics Center for Child Health Research sponsored a meeting of experts and knowledgeable stakeholders to identify 1) the special information needs of pediatric care and 2) health service research questions related to the use of information technology in children's health care. Technologies that support the care of children must address issues related to growth and development, children's changing physiology, and the unique diseases of children and interventions of pediatric care. Connectivity and data integration are particular concerns for child health care workers. Consumer health information needs for this population extend beyond the needs of one individual to the needs of the family. Recommendations of the attendees include rapid implementation of features in electronic health information systems that support pediatric care and involvement of child health experts in policy making, standards setting, education, and advocacy. A proposed research agenda should address both effectiveness and costs of information technology, with special consideration for the needs of children, the development and evaluation of clinical decision support in pediatric settings, understanding of the epidemiology of iatrogenic injury in childhood, supplementation of vocabulary standards with pediatrics-specific terminology, and improvement in health care access for children, using telemedicine.

An approach to guideline implementation with GEM.

Implementation of practice guidelines refers to the creation of strategies and systems to operationalize the knowledge and recommendations set forth by guideline developers. We describe an approach to guideline implementation that makes direct use of the guideline document as a knowledge base. The Guideline Elements Model (GEM) provides an XML-based guideline document model that facilitates implementation of guidelines. Knowledge extraction using GEM requires document markup rather than programming and can promote authenticity and consistent knowledge encoding. Knowledge customization for the local enterprise requires addition of meta-information to pertinent components of the GEM hierarchy in a design database. GEM provides an audit trail to track local adaptation. Knowledge integration with patient data can be promoted using information management services. A design goal is to devise a system that can be applied by local clinical domain experts, quality assurance experts, and information systems programmers without requiring trained informaticians and knowledge engineers to serve as intermediaries

Evaluation of guideline quality using GEM-Q.

A variety of rating instruments that evaluate the quality of practice guidelines have been published. Application of these instruments can be difficult and time-consuming. In a literature review, we identified two evaluation instruments that are comprehensive, have clearly defined constructs, and have undergone validation/testing-the Guidelines Quality Assessment Questionnaire (GQAQ) and the Appraisal Instrument for Clinical Guidelines (AICG). Overall, the AICG is more comprehensive. The AICG addresses the implementability of a guideline, which is not evaluated by the GQAQ. However, the GQAQ is more amenable to computerization. GEM-Q is a Guideline Elements Model (GEM)-derived application intended to facilitate automated evaluation of guideline quality using one of the published instruments. To develop GEM-Q, various items in the GQAQ were mapped to corresponding elements in the GEM hierarchy and a customized XSL stylesheet was designed based on this mapping. GEM-Q selectively extracts text components of the guideline relevant to quality evaluation and displays the results in HTML format. GEM-Q was applied to a set of six guidelines to test its reliability. It ranked two guidelines as of "good" quality, two as "intermediate", and two as "poor". In all six instances, GEM-Q ranked guidelines in the same order of quality as the experts who validated the GQAQ. This work demonstrates the feasibility of developing an application to facilitate automated guideline quality evaluation.

Using GEM-encoded guidelines to generate medical logic modules.

Among the most effective strategies for changing the process and outcomes of clinical care are those that make use of computer-mediated decision support. A variety of representation models that facilitate computer-based implementation of medical knowledge have been published, including the Guideline Elements Model (GEM) and the Arden Syntax for Medical Logic Modules (MLMs). We describe an XML-based application that facilitates automated generation of partially populated MLMs from GEM-encoded guidelines. These MLMs can be further edited and shared among Arden-compliant information systems to provide decision support. Our work required three steps: (a) Knowledge extraction from published guideline documents using GEM, (b) Mapping GEM elements to the MLM slots, and (c) XSL transformation of the GEM-encoded guideline. Processing of a sample guideline generated 15 MLMs, each corresponding to a conditional or imperative element in the GEM structure. Mechanisms for linking various MLMs are necessary to represent the complexity of logic typical of a guideline.

An implementation framework for GEM encoded guidelines.

Access to timely decision support information is critical for delivery of high-quality medical care. Transformation of clinical knowledge that is originally expressed in the form of a guideline to a computable format is one of the main obstacles to the integration of knowledge sharing functionality into computerized clinical systems. The Guideline Element Model (GEM) provides a methodology for such a transformation. Although the model has been used to store heterogeneous guideline knowledge, it is important to demonstrate that GEM markup facilitates guideline implementation. This report demonstrates the feasibility of implementation of GEM-encoded guideline recommendations using Apache Group s Cocoon Web Publishing Framework. We further demonstrate how XML-based programming allows for maintaining the separation of guideline content from processing logic and from presentation format. Finally, we analyze whether the guideline authors original intent has been sufficiently captured and conveyed to the end user.

A preliminary evaluation of guideline content mark-up using GEM--an XML guideline elements model.

OBJECTIVE: To describe application of GEM to analysis and categorization of guideline content. METHOD: We examined the application of GEM constructs to the AAP guideline on neurodiagnostic evaluation of febrile seizures. Subjects at 4 sites marked-up the guideline content using a hierarchical template that includes branches for identity, developer, purpose, intended audience, method of development, knowledge components, testing, and review. The types of elements used were tabulated. Subjects were surveyed regarding the usability of the model. RESULTS: Eight subjects analyzed the guideline, using between 46 and 149 elements to model its content. There was considerable variation in the application of elements. The number of elements used correlated with time to complete the task. Subjects found application of GEM to be straightforward in 6 of 8 categories and sufficiently comprehensive to model the guideline's information content. CONCLUSIONS: Subjects found GEM constructs were able to model the content of the guideline. Improved editing tools will facilitate translation.

GEM: a proposal for a more comprehensive guideline document model using XML.

OBJECTIVE: To develop a guideline document model that includes a sufficiently broad set of concepts to be useful throughout the guideline life cycle. DESIGN: Current guideline document models are limited in that they reflect the specific orientation of the stakeholder who created them; thus, developers and disseminators often provide few constructs for conceptualizing recommendations, while implementers de-emphasize concepts related to establishing guideline validity. The authors developed the Guideline Elements Model (GEM) using XML to better represent the heterogeneous knowledge contained in practice guidelines. Core constructs were derived from the Institute of Medicine's Guideline Appraisal Instrument, the National Guideline Clearinghouse, and the augmented decision table guideline representation. These were supplemented by additional concepts from a literature review. RESULTS: The GEM hierarchy includes more than 100 elements. Major concepts relate to a guideline's identity, developer, purpose, intended audience, method of development, target population, knowledge components, testing, and review plan. Knowledge components in guideline documents include recommendations (which in turn comprise conditionals and imperatives), definitions, and algorithms. CONCLUSION: GEM is more comprehensive than existing models and is expressively adequate to represent the heterogeneous information contained in guidelines. Use of XML contributes to a flexible, comprehensible, shareable, and reusable knowledge representation that is both readable by human beings and processible by computers.

Continuous low-dose antibiotics and cerclage for recurrent second-trimester pregnancy loss.

OBJECTIVE: To evaluate the role of continuous low-dose antibiotics in the prolongation of pregnancy in women at risk for recurrent second-trimester loss. METHODS: Only patients with a history of previous recurrent second-trimester losses associated with failed cervical cerclages were prospectively included in the study. Patients received low-dose antibiotics until delivery. Cerclage was performed at 14-24 weeks' gestation on the basis of transvaginal sonographic findings of cervical funneling. Outcome was evaluated by weeks of pregnancy gained in the current pregnancy as compared to the previous pregnancy. RESULTS: Ten patients were eligible for study after exclusions. All 10 achieved fetal viability. Pregnancy was prolonged by a mean of 13.4 +/- 4.2 weeks beyond the previous pregnancy. This was highly statistically significant (P < .001). CONCLUSION: Continuous low-dose antibiotics prolonged pregnancy in patients with recurrent second-trimester pregnancy losses and prior failed cerclage. Randomized clinical trials are needed to confirm the role of antibiotics in these high-risk pregnancies.

A guideline implementation system using handheld computers for office management of asthma: effects on adherence and patient outcomes.

OBJECTIVE: To evaluate effects on the process and outcomes of care brought about by use of a handheld, computer-based system that implements the American Academy of Pediatrics guideline on office management of asthma exacerbations. DESIGN: A before-after trial with randomly selected, office-based Connecticut pediatricians. In both the control and intervention phases, physicians collected data from 10 patient encounters for acute asthma exacerbations. During the intervention phase, the computer provided for structured encounter documentation and offered recommendations based on the guideline of the American Academy of Pediatrics. Patients were contacted by telephone 7 to 14 days after the visit to assess outcomes. RESULTS: Nine study-physicians enrolled 91 patients in the control phase and 74 in the intervention phase. Follow-up information was available for 93% of encounters. Use of the intervention was associated with increased mean frequency/visit of: 1) measurements of peak expiratory flow rate (2.18 vs 1.57) and oxygen saturation (1.12 vs.42), and 2) administration of nebulized beta2-agonists (1.25 vs.71). Visits in the intervention phase lasted longer and fees were higher ($145.61 vs $103.11). There were no significant differences in immediate disposition or subsequent emergency department visits, hospitalizations, missed school, or caretaker's missed work during the 7 days post visit. CONCLUSION: Use of handheld computers that provide guideline-based decision support was associated with increased physician adherence to guideline recommendations; however, visits were prolonged, fees were higher, and no improvement could be demonstrated with regard to the observed intermediate-term patient outcomes. Guideline implementers (and users) should be cautious about putting unvalidated recommendations into practice.

Assessing growth patterns--routine but sometimes overlooked.

We performed a retrospective chart review for 149 randomly selected well-child encounters to evaluate the frequency with which clinicians plot growth measurements and document growth abnormalities during health maintenance visits. Providers failed to plot at least one measurement of height, weight, and/or head circumference in 31 of 149 encounters (21%, 95% CI = 14.5% to 27.5%). Growth abnormalities in size, velocity, and/or disproportion were not documented in 22 of 40 relevant encounters (55%, 95% CI = 40% to 70%). Overall, 52 of 149 encounters (35%) were associated with an unplotted measurement and/or an undocumented growth abnormality. We feel that both documentation and assessment of growth represent potential areas for quality improvement.

User satisfaction and frustration with a handheld, pen-based guideline implementation system for asthma.

OBJECTIVE: To evaluate clinicians' satisfaction and frustrations with the use of a handheld computer system that implements a guideline for management of childhood asthma exacerbations. SETTING: Nine primary-care pediatric practices. DESIGN: Survey component of a randomized, prospective before-after trial. INTERVENTION: Newton MessagePad outfitted with custom software (called "AsthMonitor") that assists in documentation of clinical findings and provides guideline-based recommendations. RESULTS: Overall, 3 users gave strongly positive global ratings while 6 users were neutral. The majority used the documentation functions concurrently with care. Except for recommendations to administer oxygen (which were unsupported by evidence), users found the recommendations appropriate and appreciated the reminders. Seven of 9 participants believed it took more time to document with AsthMonitor. CONCLUSIONS: Handheld computers are acceptable to some office-based practitioners to provide guideline-based advice within the context of the clinical encounter.

American Academy of Pediatrics. Pediatric Practice Action Group and Task Force on Medical Informatics. Privacy protection and health information: patient rights and pediatrician responsibilities.

Pediatricians and pediatric medical and surgical subspecialists should know their legal responsibilities to protect the privacy of identifiable patient health information. Although paper and electronic medical records have the same privacy standards, health data that are stored or transmitted electronically are vulnerable to unique security breaches. This statement describes the privacy and confidentiality needs and rights of pediatric patients and suggests appropriate security strategies to deter unauthorized access and inappropriate use of patient data. Limitations to physician liability are discussed for transferred data. Any new standards for patient privacy and confidentiality must balance the health needs of the community and the rights of the patient without compromising the ability of pediatricians to provide quality care.

An object-oriented framework for the development of computer-based guideline implementations.

Clinical practice guidelines provide a means of directing medical care towards clinically appropriate and cost-effective interventions. A direct relationship exists between the integration of a guideline into clinical workflow and the effectiveness of the guideline in influencing clinicians' behavior. Computer-based guideline implementations, used at the point-of-care, accomplish this integration. Employing object-oriented technologies, we propose a framework of reusable components for the development of guideline implementation systems. We have identified eight information management services that are common to such systems. Our framework integrates these services and their respective reusable components into clinical workflow to promote the development of comprehensive guideline implementation systems, which should ultimately enhance guideline compliance and the overall quality of care.

Computer-based guideline implementation systems: a systematic review of functionality and effectiveness.

In this systematic review, the authors analyze the functionality provided by recent computer-based guideline implementation systems and characterize the effectiveness of the systems. Twenty-five studies published between 1992 and January 1998 were identified. Articles were included if the authors indicated an intent to implement guideline recommendations for clinicians and if the effectiveness of the system was evaluated. Provision of eight information management services and effects on guideline adherence, documentation, user satisfaction, and patient outcome were noted. All systems provided patient-specific recommendations. In 19, recommendations were available concurrently with care. Explanation services were described for nine systems. Nine systems allowed interactive documentation, and 17 produced paper-based output. Communication services were present most often in systems integrated with electronic medical records. Registration, calculation, and aggregation services were infrequently reported. There were 10 controlled trials (9 randomized) and 10 time-series correlational studies. Guideline adherence improved in 14 of 18 systems in which it was measured. Documentation improved in 4 of 4 studies.

A design model for computer-based guideline implementation based on information management services.

Clinical practice guidelines must be implemented effectively if they are to influence the behavior of clinicians. The authors describe a model for computer-based guideline implementation that identifies eight information management services needed to integrate guideline-based decision support with clinical workflow. Recommendation services determine appropriate activities in specific clinical circumstances. Documentation services involve data capture. Registration services integrate demographic and administrative data. Explanation services enhance the credibility of automated recommendations by providing supportive evidence and rating the quality of evidence. Calculation services measure time intervals, suggest medication dosages, and perform other computational tasks. Communication services employ standards for information transfer and provide data security. Effective presentation services facilitate understanding of complex data, clarify trends, and format written materials (including prescriptions) for patients. Aggregation services associate outcomes with specific guideline interventions. The authors provide examples of the eight services that make up the model from five evidence-based practice parameters developed by the American Academy of Pediatrics.

Transition to a computer-based record using scannable, structured encounter forms.

OBJECTIVE: To evaluate the quality of documentation and user satisfaction with a structured documentation system for pediatric health maintenance encounters, using scanned paper-based forms to generate an electronic medical record. DESIGN: (1) A retrospective medical record review comparing 16 structured (ST) records with 16 contemporaneously created unstructured records, (2) a questionnaire evaluation of user satisfaction, and (3) an electronic records review of patients seen 1 year following the full implementation of the system to evaluate persistence of the effect. SETTING: The Yale-New Haven Hospital Pediatric Primary Care Center, New Haven, Conn, an inner-city clinic in an academic center. PARTICIPANTS: (1) A random sample of 16 health maintenance records completed by first- and second-year residents in February 1996 matched for patient's age and provider training level with 16 contemporaneously documented visits, (2) 16 of 18 pediatric level 1 residents and 14 of 16 pediatric level 2 residents who completed questionnaires, and (3) all electronic records of health maintenance visits during February 1997. MAIN OUTCOME MEASURES: The number of data elements documented and the percentage of records that record specific components of the health maintenance encounter. User satisfaction was specified on a Likert scale. RESULTS: Overall, residents in the ST records group documented more data elements per visit than did those in the unstructured records group. The number of developmental items documented was 11.5 per visit in the ST records group and 4.8 per visit in the unstructured records group (P = .004). Likewise, anticipatory guidance was more thoroughly documented in the ST records group--8.3 items per visit vs 2.5 items per visit (P < .001). Ninety percent of the users preferred the ST records. One year after the adoption of the ST recording system, high levels of thoroughness persisted. CONCLUSIONS: Structured, scannable encounter forms can facilitate documentation of patient care and are well accepted by users. They can provide an effective mechanism to ease the transition to a computer-based patient record.