"Covering provider": an effort to streamline clinical communication chaos.

Objective: This report describes a root cause analysis of incorrect provider assignments and a standardized workflow developed to improve the clarity and accuracy of provider assignments. Materials and Methods: A multidisciplinary working group involving housestaff was assembled. Key drivers were identified using value stream mapping and fishbone analysis. A report was developed to allow for the analysis of correct provider assignments. A standardized workflow was created and piloted with a single service line. Pre- and post-pilot surveys were administered to nursing staff and participating housestaff on the unit. Results: Four key drivers were identified. A standardized workflow was created with an exclusive treatment team role in Epic held by a single provider at any given time, with a corresponding patient list column displaying provider information for each patient. Pre- and post-survey responses report decreased confusion, decreased provider identification errors, and increased user satisfaction among RNs and residents with sustained uptake over time. Conclusion: This work demonstrates structured root cause analysis, notably engaging housestaff, to develop a standardized workflow for an understudied and growing problem. The development of tools and strategies to address the widespread burdens resulting from clinical communication failures is needed.

Considerations in the reliability and fairness audits of predictive models for advance care planning.

Multiple reporting guidelines for artificial intelligence (AI) models in healthcare recommend that models be audited for reliability and fairness. However, there is a gap of operational guidance for performing reliability and fairness audits in practice. Following guideline recommendations, we conducted a reliability audit of two models based on model performance and calibration as well as a fairness audit based on summary statistics, subgroup performance and subgroup calibration. We assessed the Epic End-of-Life (EOL) Index model and an internally developed Stanford Hospital Medicine (HM) Advance Care Planning (ACP) model in 3 practice settings: Primary Care, Inpatient Oncology and Hospital Medicine, using clinicians' answers to the surprise question ("Would you be surprised if [patient X] passed away in [Y years]?") as a surrogate outcome. For performance, the models had positive predictive value (PPV) at or above 0.76 in all settings. In Hospital Medicine and Inpatient Oncology, the Stanford HM ACP model had higher sensitivity (0.69, 0.89 respectively) than the EOL model (0.20, 0.27), and better calibration (O/E 1.5, 1.7) than the EOL model (O/E 2.5, 3.0). The Epic EOL model flagged fewer patients (11%, 21% respectively) than the Stanford HM ACP model (38%, 75%). There were no differences in performance and calibration by sex. Both models had lower sensitivity in Hispanic/Latino male patients with Race listed as "Other." 10 clinicians were surveyed after a presentation summarizing the audit. 10/10 reported that summary statistics, overall performance, and subgroup performance would affect their decision to use the model to guide care; 9/10 said the same for overall and subgroup calibration. The most commonly identified barriers for routinely conducting such reliability and fairness audits were poor demographic data quality and lack of data access. This audit required 115 person-hours across 8-10 months. Our recommendations for performing reliability and fairness audits include verifying data validity, analyzing model performance on intersectional subgroups, and collecting clinician-patient linkages as necessary for label generation by clinicians. Those responsible for AI models should require such audits before model deployment and mediate between model auditors and impacted stakeholders.

Building a Learning Health System: Creating an Analytical Workflow for Evidence Generation to Inform Institutional Clinical Care Guidelines.

BACKGROUND: One key aspect of a learning health system (LHS) is utilizing data generated during care delivery to inform clinical care. However, institutional guidelines that utilize observational data are rare and require months to create, making current processes impractical for more urgent scenarios such as those posed by the COVID-19 pandemic. There exists a need to rapidly analyze institutional data to drive guideline creation where evidence from randomized control trials are unavailable. OBJECTIVES: This article provides a background on the current state of observational data generation in institutional guideline creation and details our institution's experience in creating a novel workflow to (1) demonstrate the value of such a workflow, (2) demonstrate a real-world example, and (3) discuss difficulties encountered and future directions. METHODS: Utilizing a multidisciplinary team of database specialists, clinicians, and informaticists, we created a workflow for identifying and translating a clinical need into a queryable format in our clinical data warehouse, creating data summaries and feeding this information back into clinical guideline creation. RESULTS: Clinical questions posed by the hospital medicine division were answered in a rapid time frame and informed creation of institutional guidelines for the care of patients with COVID-19. The cost of setting up a workflow, answering the questions, and producing data summaries required around 300 hours of effort and $300,000 USD. CONCLUSION: A key component of an LHS is the ability to learn from data generated during care delivery. There are rare examples in the literature and we demonstrate one such example along with proposed thoughts of ideal multidisciplinary team formation and deployment.

Opportunities for addressing gaps in primary care shared decision-making with technology: a mixed-methods needs assessment.

OBJECTIVES: To analyze current practices in shared decision-making (SDM) in primary care and perform a needs assessment for the role of information technology (IT) interventions. MATERIALS AND METHODS: A mixed-methods study was conducted in three phases: (1) ethnographic observation of clinical encounters, (2) patient interviews, and (3) physician interviews. SDM was measured using the validated OPTION scale. Semistructured interviews followed an interview guide (developed by our multidisciplinary team) informed by the Traditional Decision Conflict Scale and Shared Decision Making Questionnaire. Field notes were independently coded and analyzed by two reviewers in Dedoose. RESULTS: Twenty-four patient encounters were observed in 3 diverse practices with an average OPTION score of 57.2 (0-100 scale; 95% confidence interval [CI], 51.8-62.6). Twenty-two patient and 8 physician interviews were conducted until thematic saturation was achieved. Cohen's kappa, measuring coder agreement, was 0.42. Patient domains were: establishing trust, influence of others, flexibility, frustrations, values, and preferences. Physician domains included frustrations, technology (concerns, existing use, and desires), and decision making (current methods used, challenges, and patients' understanding). DISCUSSION: Given low SDM observed, multiple opportunities for technology to enhance SDM exist based on specific OPTION items that received lower scores, including: (1) checking the patient's preferred information format, (2) asking the patient's preferred level of involvement in decision making, and (3) providing an opportunity for deferring a decision. Based on data from interviews, patients and physicians value information exchange and are open to technologies that enhance communication of care options. CONCLUSION: Future primary care IT platforms should prioritize the 3 quantitative gaps identified to improve physician-patient communication and relationships. Additionally, SDM tools should seek to standardize common workflow steps across decisions and focus on barriers to increasing adoption of effective SDM tools into routine primary care.