Best practices for preventing malfunctions in rule-based clinical decision support alerts and reminders: Results of a Delphi study.

OBJECTIVE: Developing effective and reliable rule-based clinical decision support (CDS) alerts and reminders is challenging. Using a previously developed taxonomy for alert malfunctions, we identified best practices for developing, testing, implementing, and maintaining alerts and avoiding malfunctions. MATERIALS AND METHODS: We identified 72 initial practices from the literature, interviews with subject matter experts, and prior research. To refine, enrich, and prioritize the list of practices, we used the Delphi method with two rounds of consensus-building and refinement. We used a larger than normal panel of experts to include a wide representation of CDS subject matter experts from various disciplines. RESULTS: 28 experts completed Round 1 and 25 completed Round 2. Round 1 narrowed the list to 47 best practices in 7 categories: knowledge management, designing and specifying, building, testing, deployment, monitoring and feedback, and people and governance. Round 2 developed consensus on the importance and feasibility of each best practice. DISCUSSION: The Delphi panel identified a range of best practices that may help to improve implementation of rule-based CDS and avert malfunctions. Due to limitations on resources and personnel, not everyone can implement all best practices. The most robust processes require investing in a data warehouse. Experts also pointed to the issue of shared responsibility between the healthcare organization and the electronic health record vendor. CONCLUSION: These 47 best practices represent an ideal situation. The research identifies the balance between importance and difficulty, highlights the challenges faced by organizations seeking to implement CDS, and describes several opportunities for future research to reduce alert malfunctions.

Clinical decision support alert malfunctions: analysis and empirically derived taxonomy.

Objective: To develop an empirically derived taxonomy of clinical decision support (CDS) alert malfunctions. Materials and Methods: We identified CDS alert malfunctions using a mix of qualitative and quantitative methods: (1) site visits with interviews of chief medical informatics officers, CDS developers, clinical leaders, and CDS end users; (2) surveys of chief medical informatics officers; (3) analysis of CDS firing rates; and (4) analysis of CDS overrides. We used a multi-round, manual, iterative card sort to develop a multi-axial, empirically derived taxonomy of CDS malfunctions. Results: We analyzed 68 CDS alert malfunction cases from 14 sites across the United States with diverse electronic health record systems. Four primary axes emerged: the cause of the malfunction, its mode of discovery, when it began, and how it affected rule firing. Build errors, conceptualization errors, and the introduction of new concepts or terms were the most frequent causes. User reports were the predominant mode of discovery. Many malfunctions within our database caused rules to fire for patients for whom they should not have (false positives), but the reverse (false negatives) was also common. Discussion: Across organizations and electronic health record systems, similar malfunction patterns recurred. Challenges included updates to code sets and values, software issues at the time of system upgrades, difficulties with migration of CDS content between computing environments, and the challenge of correctly conceptualizing and building CDS. Conclusion: CDS alert malfunctions are frequent. The empirically derived taxonomy formalizes the common recurring issues that cause these malfunctions, helping CDS developers anticipate and prevent CDS malfunctions before they occur or detect and resolve them expediently.

Studying Readiness for Clinical Decision Support for Worker Health Using the Rapid Assessment Process and Mixed Methods Interviews.

To determine how the Rapid Assessment Process (RAP) can be adapted to evaluate the readiness of primary care clinics for acceptance and use of computerized clinical decision support (CDS) related to clinical management of working patients, we used a unique blend of ethnographic methods for gathering data. First, knowledge resources, which were prototypes of CDS content areas (diabetes, lower back pain, and asthma) containing evidence-based information, decision logic, scenarios and examples of use, were developed by subject matter experts. A team of RAP researchers then visited five clinic settings to identify barriers and facilitators to implementing CDS about the health of workers in general and the knowledge resources specifically. Methods included observations, semi-structured qualitative interviews and graphic elicitation interviews about the knowledge resources. We used both template and grounded hermeneutic approaches to data analysis. Preliminary results indicate that the methods succeeded in generating specific actionable recommendations for CDS design.

B cells regulate macrophage phenotype and response to chemotherapy in squamous carcinomas.

B cells foster squamous cell carcinoma (SCC) development through deposition of immunoglobulin-containing immune complexes in premalignant tissue and Fcγ receptor-dependent activation of myeloid cells. Because human SCCs of the vulva and head and neck exhibited hallmarks of B cell infiltration, we examined B cell-deficient mice and found reduced support for SCC growth. Although ineffective as a single agent, treatment of mice bearing preexisting SCCs with B cell-depleting αCD20 monoclonal antibodies improved response to platinum- and Taxol-based chemotherapy. Improved chemoresponsiveness was dependent on altered chemokine expression by macrophages that promoted tumor infiltration of activated CD8(+) lymphocytes via CCR5-dependent mechanisms. These data reveal that B cells, and the downstream myeloid-based pathways they regulate, represent tractable targets for anticancer therapy in select tumors.