Factors Associated with Acute Telemental Health Consultations in Older Veterans.

Introduction: The United States Veterans Health Administration is a leader in the use of telemental health (TMH) to enhance access to mental healthcare amidst a nationwide shortage of mental health professionals. The Tennessee Valley Veterans Affairs (VA) Health System piloted TMH in its emergency department (ED) and urgent care clinic (UCC) in 2019, with full 24/7 availability beginning March 1, 2020. Following implementation, preliminary data demonstrated that veterans ≥65 years old were less likely to receive TMH than younger patients. We sought to examine factors associated with older veterans receiving TMH consultations in acute, unscheduled, outpatient settings to identify limitations in the current process. Methods: This was a retrospective cohort study conducted within the Tennessee Valley VA Health System. We included veterans ≥55 years who received a mental health consultation in the ED or UCC from April 1, 2020-September 30, 2022. Telemental health was administered by a mental health clinician (attending physician, resident physician, nurse practitioner, or physician assistant) via iPad, whereas in-person evaluations were performed in the ED. We examined the influence of patient demographics, visit timing, chief complaint, and psychiatric history on TMH, using multivariable logistic regression. Results: Of the 254 patients included in this analysis, 177 (69.7%) received TMH. Veterans with high-risk chief complaints (suicidal ideation, homicidal ideation, or agitation) were less likely to receive TMH consultation (adjusted odds ratio [AOR]: 0.47, 95% confidence interval [CI] 0.24-0.95). Compared to attending physicians, nurse practitioners and physician assistants were associated with increased TMH use (AOR 4.81, 95% CI 2.04-11.36), whereas consultation by resident physicians was associated with decreased TMH use (AOR 0.04, 95% CI 0.00-0.59). The UCC used TMH for all but one encounter. Patient characteristics including their visit timing, gender, additional medical complaints, comorbidity burden, and number of psychoactive medications did not influence use of TMH. Conclusion: High-risk chief complaints, location, and type of mental health clinician may be key determinants of telemental health use in older adults. This may help expand mental healthcare access to areas with a shortage of mental health professionals and prevent potentially avoidable transfers in low-acuity situations. Further studies and interventions may optimize TMH for older patients to ensure safe, equitable mental health care.

Use of noisy labels as weak learners to identify incompletely ascertainable outcomes: A Feasibility study with opioid-induced respiratory depression.

Objective: Assigning outcome labels to large observational data sets in a timely and accurate manner, particularly when outcomes are rare or not directly ascertainable, remains a significant challenge within biomedical informatics. We examined whether noisy labels generated from subject matter experts' heuristics using heterogenous data types within a data programming paradigm could provide outcomes labels to a large, observational data set. We chose the clinical condition of opioid-induced respiratory depression for our use case because it is rare, has no administrative codes to easily identify the condition, and typically requires at least some unstructured text to ascertain its presence. Materials and methods: Using de-identified electronic health records of 52,861 post-operative encounters, we applied a data programming paradigm (implemented in the Snorkel software) for the development of a machine learning classifier for opioid-induced respiratory depression. Our approach included subject matter experts creating 14 labeling functions that served as noisy labels for developing a probabilistic Generative model. We used probabilistic labels from the Generative model as outcome labels for training a Discriminative model on the source data. We evaluated performance of the Discriminative model with a hold-out test set of 599 independently-reviewed patient records. Results: The final Discriminative classification model achieved an accuracy of 0.977, an F1 score of 0.417, a sensitivity of 1.0, and an AUC of 0.988 in the hold-out test set with a prevalence of 0.83% (5/599). Discussion: All of the confirmed Cases were identified by the classifier. For rare outcomes, this finding is encouraging because it reduces the number of manual reviews needed by excluding visits/patients with low probabilities. Conclusion: Application of a data programming paradigm with expert-informed labeling functions might have utility for phenotyping clinical phenomena that are not easily ascertainable from highly-structured data.

Use of Noisy Labels as Weak Learners to Identify Incompletely Ascertainable Outcomes: A Feasibility Study with Opioid-Induced Respiratory Depression.

Objective: Assigning outcome labels to large observational data sets in a timely and accurate manner, particularly when outcomes are rare or not directly ascertainable, remains a significant challenge within biomedical informatics. We examined whether noisy labels generated from subject matter experts' heuristics using heterogenous data types within a data programming paradigm could provide outcomes labels to a large, observational data set. We chose the clinical condition of opioid-induced respiratory depression for our use case because it is rare, has no administrative codes to easily identify the condition, and typically requires at least some unstructured text to ascertain its presence. Materials and Methods: Using de-identified electronic health records of 52,861 post-operative encounters, we applied a data programming paradigm (implemented in the Snorkel software) for the development of a machine learning classifier for opioid-induced respiratory depression. Our approach included subject matter experts creating 14 labeling functions that served as noisy labels for developing a probabilistic Generative model. We used probabilistic labels from the Generative model as outcome labels for training a Discriminative model on the source data. We evaluated performance of the Discriminative model with a hold-out test set of 599 independently-reviewed patient records. Results: The final Discriminative classification model achieved an accuracy of 0.977, an F1 score of 0.417, a sensitivity of 1.0, and an AUC of 0.988 in the hold-out test set with a prevalence of 0.83% (5/599). Discussion: All of the confirmed Cases were identified by the classifier. For rare outcomes, this finding is encouraging because it reduces the number of manual reviews needed by excluding visits/patients with low probabilities. Conclusion: Application of a data programming paradigm with expert-informed labeling functions might have utility for phenotyping clinical phenomena that are not easily ascertainable from highly-structured data.

Developing a Quality Improvement Implementation Taxonomy for Organizational Employee Wellness Initiatives.

BACKGROUND: Standardized taxonomies (STs) facilitate knowledge representation and semantic interoperability within health care provision and research. However, a gap exists in capturing knowledge representation to classify, quantify, qualify, and codify the intersection of evidence and quality improvement (QI) implementation. This interprofessional case report leverages a novel semantic and ontological approach to bridge this gap. OBJECTIVES: This report had two objectives. First, it aimed to synthesize implementation barrier and facilitator data from employee wellness QI initiatives across Veteran Affairs health care systems through a semantic and ontological approach. Second, it introduced an original framework of this use-case-based taxonomy on implementation barriers and facilitators within a QI process. METHODS: We synthesized terms from combined datasets of all-site implementation barriers and facilitators through QI cause-and-effect analysis and qualitative thematic analysis. We developed the Quality Improvement and Implementation Taxonomy (QIIT) classification scheme to categorize synthesized terms and structure. This framework employed a semantic and ontological approach. It was built upon existing terms and models from the QI Plan, Do, Study, Act phases, the Consolidated Framework for Implementation Research domains, and the fishbone cause-and-effect categories. RESULTS: The QIIT followed a hierarchical and relational classification scheme. Its taxonomy was linked to four QI Phases, five Implementing Domains, and six Conceptual Determinants modified by customizable Descriptors and Binary or Likert Attribute Scales. CONCLUSION: This case report introduces a novel approach to standardize the process and taxonomy to describe evidence translation to QI implementation barriers and facilitators. This classification scheme reduces redundancy and allows semantic agreements on concepts and ontological knowledge representation. Integrating existing taxonomies and models enhances the efficiency of reusing well-developed taxonomies and relationship modeling among constructs. Ultimately, employing STs helps generate comparable and sharable QI evaluations for forecast, leading to sustainable implementation with clinically informed innovative solutions.

Developing a Quality Improvement Implementation Taxonomy for Organizational Employee Wellness Initiatives.

BACKGROUND: Standardized taxonomies (STs) facilitate knowledge representation and semantic interoperability within health care provision and research. However, a gap exists in capturing knowledge representation to classify, quantify, qualify, and codify the intersection of evidence and quality improvement (QI) implementation. This interprofessional case report leverages a novel semantic and ontological approach to bridge this gap. OBJECTIVES: This report had two objectives. First, it aimed to synthesize implementation barrier and facilitator data from employee wellness QI initiatives across Veteran Affairs health care systems through a semantic and ontological approach. Second, it introduced an original framework of this use-case-based taxonomy on implementation barriers and facilitators within a QI process. METHODS: We synthesized terms from combined datasets of all-site implementation barriers and facilitators through QI cause-and-effect analysis and qualitative thematic analysis. We developed the Quality Improvement and Implementation Taxonomy (QIIT) classification scheme to categorize synthesized terms and structure. This framework employed a semantic and ontological approach. It was built upon existing terms and models from the QI Plan, Do, Study, Act phases, the Consolidated Framework for Implementation Research domains, and the fishbone cause-and-effect categories. RESULTS: The QIIT followed a hierarchical and relational classification scheme. Its taxonomy was linked to four QI Phases, five Implementing Domains, and six Conceptual Determinants modified by customizable Descriptors and Binary or Likert Attribute Scales. CONCLUSION: This case report introduces a novel approach to standardize the process and taxonomy to describe evidence translation to QI implementation barriers and facilitators. This classification scheme reduces redundancy and allows semantic agreements on concepts and ontological knowledge representation. Integrating existing taxonomies and models enhances the efficiency of reusing well-developed taxonomies and relationship modeling among constructs. Ultimately, employing STs helps generate comparable and sharable QI evaluations for forecast, leading to sustainable implementation with clinically informed innovative solutions.

Inpatient nurses' preferences and decisions with risk information visualization.

OBJECTIVE: We examined the influence of 4 different risk information formats on inpatient nurses' preferences and decisions with an acute clinical deterioration decision-support system. MATERIALS AND METHODS: We conducted a comparative usability evaluation in which participants provided responses to multiple user interface options in a simulated setting. We collected qualitative data using think aloud methods. We collected quantitative data by asking participants which action they would perform after each time point in 3 different patient scenarios. RESULTS: More participants (n = 6) preferred the probability format over relative risk ratios (n = 2), absolute differences (n = 2), and number of persons out of 100 (n = 0). Participants liked average lines, having a trend graph to supplement the risk estimate, and consistent colors between trend graphs and possible actions. Participants did not like too much text information or the presence of confidence intervals. From a decision-making perspective, use of the probability format was associated with greater concordance in actions taken by participants compared to the other 3 risk information formats. DISCUSSION: By focusing on nurses' preferences and decisions with several risk information display formats and collecting both qualitative and quantitative data, we have provided meaningful insights for the design of clinical decision-support systems containing complex quantitative information. CONCLUSION: This study adds to our knowledge of presenting risk information to nurses within clinical decision-support systems. We encourage those developing risk-based systems for inpatient nurses to consider expressing risk in a probability format and include a graph (with average line) to display the patient's recent trends.

Detecting Problematic Opioid Use in the Electronic Health Record: Automation of the Addiction Behaviors Checklist in a Chronic Pain Population.

Importance: Individuals whose chronic pain is managed with opioids are at high risk of developing an opioid use disorder. Large data sets, such as electronic health records, are required for conducting studies that assist with identification and management of problematic opioid use. Objective: Determine whether regular expressions, a highly interpretable natural language processing technique, could automate a validated clinical tool (Addiction Behaviors Checklist1) to expedite the identification of problematic opioid use in the electronic health record. Design: This cross-sectional study reports on a retrospective cohort with data analyzed from 2021 through 2023. The approach was evaluated against a blinded, manually reviewed holdout test set of 100 patients. Setting: The study used data from Vanderbilt University Medical Center's Synthetic Derivative, a de-identified version of the electronic health record for research purposes. Participants: This cohort comprised 8,063 individuals with chronic pain. Chronic pain was defined by International Classification of Disease codes occurring on at least two different days.18 We collected demographic, billing code, and free-text notes from patients' electronic health records. Main Outcomes and Measures: The primary outcome was the evaluation of the automated method in identifying patients demonstrating problematic opioid use and its comparison to opioid use disorder diagnostic codes. We evaluated the methods with F1 scores and areas under the curve - indicators of sensitivity, specificity, and positive and negative predictive value. Results: The cohort comprised 8,063 individuals with chronic pain (mean [SD] age at earliest chronic pain diagnosis, 56.2 [16.3] years; 5081 [63.0%] females; 2982 [37.0%] male patients; 76 [1.0%] Asian, 1336 [16.6%] Black, 56 [1.0%] other, 30 [0.4%] unknown race patients, and 6499 [80.6%] White; 135 [1.7%] Hispanic/Latino, 7898 [98.0%] Non-Hispanic/Latino, and 30 [0.4%] unknown ethnicity patients). The automated approach identified individuals with problematic opioid use that were missed by diagnostic codes and outperformed diagnostic codes in F1 scores (0.74 vs. 0.08) and areas under the curve (0.82 vs 0.52). Conclusions and Relevance: This automated data extraction technique can facilitate earlier identification of people at-risk for, and suffering from, problematic opioid use, and create new opportunities for studying long-term sequelae of opioid pain management.

Data Science Implementation Trends in Nursing Practice: A Review of the 2021 Literature.

OBJECTIVES: The goal of this work was to provide a review of the implementation of data science-driven applications focused on structural or outcome-related nurse-sensitive indicators in the literature in 2021. By conducting this review, we aim to inform readers of trends in the nursing indicators being addressed, the patient populations and settings of focus, and lessons and challenges identified during the implementation of these tools. METHODS: We conducted a rigorous descriptive review of the literature to identify relevant research published in 2021. We extracted data on model development, implementation-related strategies and measures, lessons learned, and challenges and stakeholder involvement. We also assessed whether reports of data science application implementations currently follow the guidelines of the Developmental and Exploratory Clinical Investigations of DEcision support systems driven by AI (DECIDE-AI) framework. RESULTS: Of 4,943 articles found in PubMed (NLM) and CINAHL (EBSCOhost), 11 were included in the final review and data extraction. Systems leveraging data science were developed for adult patient populations and were primarily deployed in hospital settings. The clinical domains targeted included mortality/deterioration, utilization/resource allocation, and hospital-acquired infections/COVID-19. The composition of development teams and types of stakeholders involved varied. Research teams more frequently reported on implementation methods than implementation results. Most studies provided lessons learned that could help inform future implementations of data science systems in health care. CONCLUSION: In 2021, very few studies report on the implementation of data science-driven applications focused on structural- or outcome-related nurse-sensitive indicators. This gap in the sharing of implementation strategies needs to be addressed in order for these systems to be successfully adopted in health care settings.

Telemental health in emergency care settings: A qualitative analysis of considerations for sustainability and spread.

OBJECTIVES: Following rapid uptake of telehealth during the COVID-19 pandemic, we examined barriers and facilitators for sustainability and spread of telemental health video (TMH-V) as policies regarding precautions from the pandemic waned. METHODS: We conducted a qualitative study using semistructured interviews and observations guided by RE-AIM. We asked four groups, local clinicians, facility leadership, Veterans, and external partners, about barriers and facilitators impacting patient willingness to engage in TMH-V (reach), quality of care (effectiveness), barriers and facilitators impacting provider uptake (adoption), possible adaptations to TMH-V (implementation), and possibilities for long-term use of TMH-V (maintenance). Interviews were recorded, transcribed, and analyzed using framework analysis. We also observed TMH-V encounters in one emergency department (ED) and one urgent care (UC) to understand how clinicians and Veterans engaged with the technology. RESULTS: We conducted 35 interviews with ED/UC clinicians and staff (n = 10), clinical and facility leadership (n = 7), Veterans (n = 5), and external partners (n = 13), January-May 2022. We completed 10 observations. All interviewees were satisfied with the TMH-V program, and interviewees highlighted increased comfort discussing difficult topics for Veterans (reach). Clinicians identified that TMH-V allowed for cross-coverage across sites as well as increased safety and flexibility for clinicians (adoption). Opportunities for improvement include alleviating technological burdens for on-site staff, electronic health record (EHR) modifications to accurately capture workload and modality (telehealth vs. in-person), and standardizing protocols to streamline communication between on-site and remote clinical staff (implementation). Finally, interviewees encouraged its spread (maintenance) and thought there was great potential for service expansion. CONCLUSIONS: Interviewees expressed support for continuing TMH-V locally and spread to other sites. Ensuring adequate infrastructure (e.g., EHR integration and technology support) and workforce capacity are key for successful spread. Given the shortage of mental health (MH) clinicians in rural settings, TMH-V represents a promising intervention to increase the access to high-quality emergency MH care.

The effect of telemental versus in-person mental health consults in the emergency department on 30-day utilization and processes of care.

OBJECTIVES: We sought to characterize how telemental health (TMH) versus in-person mental health consults affected 30-day postevaluation utilization outcomes and processes of care in Veterans presenting to the emergency department (ED) and urgent care clinic (UCC) with acute psychiatric complaints. METHODS: This exploratory retrospective cohort study was conducted in an ED and UCC located in a single Veterans Affairs system. A mental health provider administered TMH via iPad. The primary outcome was a composite of return ED/UCC visits, rehospitalizations, or death within 30 days. The following processes of care were collected during the index visit: changes to home psychiatric medications, admission, involuntary psychiatric hold placement, parenteral benzodiazepine or antipsychotic medication use, and physical restraints or seclusion. Data were abstracted from the Veterans Affairs electronic health record and the Clinical Data Warehouse. Multivariable logistic regression was performed. Adjusted odds ratios (aORs) with their 95% confidence intervals (95% CIs) were reported. RESULTS: Of the 496 Veterans in this analysis, 346 (69.8%) received TMH, and 150 (30.2%) received an in-person mental health evaluation. There was no significant difference in the primary outcome of 30-day return ED/UCC, rehospitalization, or death (aOR 1.47, 95% CI 0.87-2.49) between the TMH and in-person groups. TMH was significantly associated with increased ED/UCC length of stay (aOR 1.46, 95% CI 1.03-2.06) and decreased use of involuntary psychiatric holds (aOR 0.42, 95% CI 0.23-0.75). There were no associations between TMH and the other processes-of-care outcomes. CONCLUSIONS: TMH was not significantly associated with the 30-day composite outcome of return ED/UCC visits, rehospitalizations, and death compared with traditional in-person mental health evaluations. TMH was significantly associated with increased ED/UCC length of stay and decreased odds of placing an involuntary psychiatric hold. Future studies are required to confirm these findings and, if confirmed, explore the potential mechanisms for these associations.

Identifying problematic opioid use in electronic health record data: Are we looking in the right place?

OBJECTIVE: To examine the value of data obtained outside of regular healthcare visits (clinical communications) to detect problematic opioid use in electronic health records (EHRs). DESIGN: A retrospective cohort study. PARTICIPANTS: Chronic pain patient records in a large academic medical center. INTERVENTIONS: We compared evidence for problematic opioid use in (1) clinic notes, (2) clinical communications, and (3) full EHR data. We analyzed keyword counts and calculated concordance and Cohen's κ between data sources. MAIN OUTCOME MEASURE: Evidence of problematic opioid use in EHR defined as none, some, or high. RESULTS: Twenty-six percent of records were discordant in determination of problematic opioid use between clinical communications and clinic notes. Of these, 54 percent detected more evidence in clinical communications, and 46 percent in clinic notes. Compared to full EHR review, clinic notes exhibited higher concordance (78 percent; κ = 0.619) than clinical communications (60 percent; κ = 0.290). CONCLUSION: Clinical communications are a valuable addition to opioid EHR research.

Data Science Trends Relevant to Nursing Practice: A Rapid Review of the 2020 Literature.

BACKGROUND: The term "data science" encompasses several methods, many of which are considered cutting edge and are being used to influence care processes across the world. Nursing is an applied science and a key discipline in health care systems in both clinical and administrative areas, making the profession increasingly influenced by the latest advances in data science. The greater informatics community should be aware of current trends regarding the intersection of nursing and data science, as developments in nursing practice have cross-professional implications. OBJECTIVES: This study aimed to summarize the latest (calendar year 2020) research and applications of nursing-relevant patient outcomes and clinical processes in the data science literature. METHODS: We conducted a rapid review of the literature to identify relevant research published during the year 2020. We explored the following 16 topics: (1) artificial intelligence/machine learning credibility and acceptance, (2) burnout, (3) complex care (outpatient), (4) emergency department visits, (5) falls, (6) health care-acquired infections, (7) health care utilization and costs, (8) hospitalization, (9) in-hospital mortality, (10) length of stay, (11) pain, (12) patient safety, (13) pressure injuries, (14) readmissions, (15) staffing, and (16) unit culture. RESULTS: Of 16,589 articles, 244 were included in the review. All topics were represented by literature published in 2020, ranging from 1 article to 59 articles. Numerous contemporary data science methods were represented in the literature including the use of machine learning, neural networks, and natural language processing. CONCLUSION: This review provides an overview of the data science trends that were relevant to nursing practice in 2020. Examinations of such literature are important to monitor the status of data science's influence in nursing practice.

Data Science Methods for Nursing-Relevant Patient Outcomes and Clinical Processes: The 2019 Literature Year in Review.

Data science continues to be recognized and used within healthcare due to the increased availability of large data sets and advanced analytics. It can be challenging for nurse leaders to remain apprised of this rapidly changing landscape. In this article, we describe our findings from a scoping literature review of papers published in 2019 that use data science to explore, explain, and/or predict 15 phenomena of interest to nurses. Fourteen of the 15 phenomena were associated with at least one paper published in 2019. We identified the use of many contemporary data science methods (eg, natural language processing, neural networks) for many of the outcomes. We found many studies exploring Readmissions and Pressure Injuries. The topics of Artificial Intelligence/Machine Learning Acceptance, Burnout, Patient Safety, and Unit Culture were poorly represented. We hope that the studies described in this article help readers: (1) understand the breadth and depth of data science's ability to improve clinical processes and patient outcomes that are relevant to nurses and (2) identify gaps in the literature that are in need of exploration.

Assess, Prevent, and Manage Pain; Both Spontaneous Awakening and Breathing Trials; Choice of Analgesia/Sedation; Delirium: Assess, Prevent, and Manage; Early Mobility; Family Engagement and Empowerment Bundle Implementation: Quantifying the Association of Access to Bundle-Enhancing Supplies and Equipment.

Describe the physical environment factors (i.e., availability, accessibility) of bundle-enhancing items and the association of physical environment with bundle adherence. DESIGN: This multicenter, exploratory, cross-sectional study used data from two ICU-based randomized controlled trials that measured daily bundle adherence. Unit- and patient-level data collection occurred between 2011 and 2016. We developed hierarchical logistic regression models using Frequentist and Bayesian frameworks. SETTING: The study included 10 medical and surgical ICUs in six academic medical centers in the United States. PATIENTS: Adults with qualifying respiratory failure and/or septic shock (e.g., mechanical ventilation, vasopressor use) were included in the randomized controlled trials. INTERVENTIONS: The Awakening and Breathing trial Coordination, Delirium assessment/management, Early mobility bundle was recommended standard of care for randomized controlled trial patients and adherence tracked daily. MEASUREMENTS AND MAIN RESULTS: The primary outcome was adherence to the full bundle and the early mobility bundle component as identified from daily adherence documentation (n = 751 patient observations). Models included unit-level measures such as minimum and maximum distances to bundle-enhancing items and patient-level age, body mass index, and daily mechanical ventilation status. Some models suggested the following variables were influential: unit size (larger associated with decreased adherence), a standard walker (presence associated with increased adherence), and age (older associated with decreased adherence). In all cases, mechanical ventilation was associated with decreased bundle adherence. CONCLUSIONS: Both unit- and patient-level factors were associated with full bundle and early mobility adherence. There is potential benefit of physical proximity to essential items for Awakening and Breathing trial Coordination, Delirium assessment/management, Early mobility bundle and early mobility adherence. Future studies with larger sample sizes should explore how equipment location and availability influence practice.

Implementation Science Training and Resources for Nurses and Nurse Scientists.

PURPOSE: The purpose of this article is to describe the differences between quality improvement and implementation science, the urgency for nurses and nurse scientists to engage in implementation science, and international educational opportunities and resources for implementation science. ORGANIZING CONSTRUCT: There is a push for providing safe, effective, patient-centered, timely, efficient, and equitable health care. Implementation science plays a key role in adoption and integration of evidence-based practices to improve quality of care. METHODS: We reviewed implementation science programs, organizations, and literature to analyze the roles of nurses and nurse scientists in translating evidence into routine practice. FINDINGS: Implementation-trained nurses and nurse scientists are needed as part of multidisciplinary teams to advance implementation science because of their unique understanding of contextual barriers within nursing practice. Likewise, nurses are uniquely qualified for recognizing what implementation strategies are needed to improve nursing care across practice settings. CONCLUSIONS: Many international clinical and training resources exist and are supplied to aid interested readers in learning more about implementation science. CLINICAL RELEVANCE: Half of research evidence never reaches the clinical setting, and the other half takes 20 years to translate into clinical practice. Implementation science-trained nurses are in a position to be excellent improvers for meaningful change in practice.

Imagining an Orientation Built on Trust.

To streamline competency assessment documentation during orientation, we developed a comprehensive, three-phase plan, consisting of a tiered skills acquisition model, entrustable professional activities, and the full incorporation of Donna Wright's recommendations for initial competency development, allowing for the transition away from the traditional skills checklist (Wright, 2005). We were able to reduce orientation time and preceptor confusion while increasing orientation process satisfaction by the end of our revisions.