A Scoring Rubric for the Knowledge Section of the Systems Quality Improvement Training and Assessment Tool.

Introduction: Quality improvement (QI) competencies for health professions trainees were developed to address health care quality. Strategies to integrate QI into curricula exist, but methods for assessing interdisciplinary learners' competency are less developed. We refined the Knowledge section scoring rubric of the Systems Quality Improvement Training and Assessment Tool (SQI TAT) and examined its validity evidence. Methods: In 2017, the SQI TAT Knowledge section was expanded to cover seven core QI concepts, and the scoring rubric was refined. Three coders independently scored 35 SQI TAT Knowledge sections (18 pretests, 17 posttests). Interrater reliability was assessed by percent agreement and Cohen's kappa for individual variables and by Lin's concordance correlation for total scores for knowledge and application. Concurrent validity was assessed by comparing responses from two groups with different QI exposure and evaluating whether differences in exposure were measured. Results: Total-score interrater reliability average measures of concordance were .89 for all coders and >.70 for six of seven concept scores. The total score discriminated the two groups (p <. 05), and five of seven concept scores were higher for the group with more QI experience. Total scores were significantly higher posttest than pretest (p < .001), with improvement in posttest knowledge scores. Discussion: The SQI TAT Knowledge section provides a comprehensive assessment of QI knowledge. The scoring rubric was able to discriminate QI knowledge along a continuum. The SQI TAT Knowledge section is not linked to a clinical context, making it useful for assessing interprofessional learners and varying education levels.

A Review of Adverse Event Reports From Emergency Departments in the Veterans Health Administration.

BACKGROUND: Previous work assessing the frequency of adverse events in emergency medicine has been limited. The emergency department (ED) provides an initial point of care for millions of patients. Given the volume of patient encounters and the complexity of medical conditions treated in the ED, it is necessary to determine the system-based issues and associated contributing factors impacting patient safety. OBJECTIVES: The aim of this retrospective study were to use root cause analysis reports of adverse events occurring in Veterans Health Administration EDs to understand the range of events that were happening and to determine the primary causes of these events as well as actions to prevent them. METHODS: Retrospective safety reports from EDs from Veterans Health Administration medical centers across the nation for a 2-year period (2015-2016) were coded by event type, root cause, and recommended actions. RESULTS: One hundred forty-four cases were included for analysis. The most common adverse events were as follows: delays in care (n = 38, 26.4%), elopements (n = 21, 14.6%), suicide attempts and deaths by suicide (n = 15, 10.4%), inappropriate discharges (n = 15, 10.4%), and errors in following procedures (n = 14, 9.7%). Overall, the most common root cause categories leading to adverse events were knowledge/educational deficits (11.4%), policies/procedures needing improvement (11.1%), and lack of standardized policies/procedures (9.4%). DISCUSSION: Root cause analysis reports are a useful tool to determine the primary systems-based factors of common adverse events in the ED. Recommendations made in this article for addressing these root causes and potentially ameliorating these events will be useful to EDs and related health systems.

A Qualitative Evaluation of Caregiver Support Services Offered at the Atlanta Veterans Affairs Health Care System.

INTRODUCTION: The Department of Veterans Affairs (VA) provides health care to approximately 300,000 patients with dementia. Recognizing the critical role caregivers play in veterans' health, the Cognitive Disorders Specialty Care Education Center of Excellence (COE) at the Atlanta VA Health Care System implemented a suite of caregiver support services, including formal programs and resource linkages. We evaluated the effectiveness of these services and identified caregiver-perceived gaps in them. METHODS: We conducted 11 semistructured interviews from November 2016 through February 2017 with caregivers of veterans seen in the COE who had participated in support services. After coding transcripts, we established a codebook of 9 major themes and conducted a thematic analysis of all transcripts. RESULTS: Caregivers spoke positively of COE caregiver services that offered information on dementia, social support, an emphasis on caregiver well-being and self-efficacy, and methods for behavioral change. Gaps identified included the need for additional dementia information and practical support in such matters as advanced directives and eligibility for VA benefits. CONCLUSION: Our findings will inform future improvements to COE caregiver support services, such as an expansion of COE's caregiver educational content and capacity building of existing components such as resource referrals. These results also highlight opportunities for COE to interface with internal and external organizations to enhance existing caregiver services.

Nurse Practitioners, Physician Assistants, and Physicians Are Comparable in Managing the First Five Years of Diabetes.

BACKGROUND: Increasing use of nurse practitioners and physician assistants is a possible solution to the shortage of primary care providers in the United States, but the quality of care they provide is not well understood. METHODS: Because the scope of practice of the 3 provider types is similar in the Veterans Health Administration, we determined whether patients managed by primary care nurse practitioners, physician assistants, or physicians had similar hemoglobin A1c levels at comparable times in the natural history of diabetes. Our retrospective cohort study examined veterans with newly diagnosed diabetes in 2008, continuous primary care from 2008 to 2012, and more than 75% of primary care visits with nurse practitioner, physician assistant, or physician. RESULTS: Of the 19,238 patients, 95.3% were male, 77.7% were white, and they had a mean age 68.5 years; 14.7%, 7.1%, and 78.2% of patients were managed by nurse practitioners, physician assistants, and physicians, respectively. Median hemoglobin A1c was comparable at diagnosis (6.6%, 6.7%, 6.7%, P > .05) and after 4 years (all 6.5%, P > .5). Hemoglobin A1c levels at initiation of the first (7.5%-7.6%) and second (8.0%-8.2%) oral medications for patients of nurse practitioners and physician assistants compared with that of physicians was also similar after adjusting for patient characteristics (all P > .05). Nurse practitioners started insulin at a lower hemoglobin A1c (9.4%) than physicians (9.7%), which remained significant after adjustment (P < .05). CONCLUSIONS: At diagnosis and during 4 years of follow-up, diabetes management by nurse practitioners and physician assistants was comparable to management by physicians. The Veterans Health Administration model for roles of nurse practitioners and physician assistants may be broadly useful to help meet the demand for primary care providers in the United States.

Inpatient Glucose Values: Determining the Nondiabetic Range and Use in Identifying Patients at High Risk for Diabetes.

BACKGROUND: Many individuals with diabetes remain undiagnosed, leading to delays in treatment and higher risk for subsequent diabetes complications. Despite recommendations for diabetes screening in high-risk groups, the optimal approach is not known. We evaluated the utility of inpatient glucose levels as an opportunistic screening tool for identifying patients at high risk for diabetes. METHODS: We retrospectively examined 462,421 patients in the US Department of Veterans Affairs healthcare system, hospitalized on medical/surgical services in 2000-2010, for ≥3 days, with ≥2 inpatient random plasma glucose (RPG) measurements. All had continuity of care: ≥1 primary care visit and ≥1 glucose measurement within 2 years before hospitalization and yearly for ≥3 years after discharge. Glucose levels during hospitalization and incidence of diabetes within 3 years after discharge in patients without diabetes were evaluated. RESULTS: Patients had a mean age of 65.0 years, body mass index of 29.9 kg/m2, and were 96% male, 71% white, and 18% black. Pre-existing diabetes was present in 39.4%, 1.3% were diagnosed during hospitalization, 8.1% were diagnosed 5 years after discharge, and 51.3% were never diagnosed (NonDM). The NonDM group had the lowest mean hospital RPG value (112 mg/dL [6.2 mmol/L]). Having at least 2 RPG values >140 mg/dL (>7.8 mmol/L), the 95th percentile of NonDM hospital glucose, provided 81% specificity for identifying incident diabetes within 3 years after discharge. CONCLUSIONS: Screening for diabetes could be considered in patients with at least 2 hospital glucose values at/above the 95th percentile of the nondiabetic range (141 mg/dL [7.8 mmol/L]).

Improving Team-Based Care Coordination Delivery and Documentation in the Health Record.

A template developed at the Atlanta VAMC standardizes and captures data about care coordination components in a patient's electronic heath record.

Participation in a National Lifestyle Change Program is associated with improved diabetes Control outcomes.

AIMS: Clinical trials show lifestyle change programs are beneficial, yet large-scale, successful translation of these programs is scarce. We investigated the association between participation in the largest U.S. lifestyle change program, MOVE!, and diabetes control outcomes. METHODS: This longitudinal, retrospective cohort study used Veterans Health Administration databases of patients with diabetes who participated in MOVE! between 2005 and 2012, or met eligibility criteria (BMI ≥25kg/m2) but did not participate. Main outcomes were diabetic eye disease, renal disease, and medication intensification. RESULTS: There were 400,170 eligible patients with diabetes, including 87,366 (22%) MOVE! PARTICIPANTS: Included patients were 96% male, 77% white, with mean age 58years and BMI 34kg/m2. Controlling for baseline measurements and age, race, sex, BMI, and antidiabetes medications, MOVE! participants had lower body weight (-0.6kg), random plasma glucose (-2.8mg/dL), and HbA1c (-0.1%) at 12months compared to nonparticipants (each p<0.001). In multivariable Cox models, MOVE! participants had lower incidence of eye disease (hazard ratio 0.80, 95% CI 0.75-0.84) and renal disease (HR 0.89, 95% CI 0.86-0.92) and reduced medication intensification (HR 0.82, 95% CI 0.80-0.84). CONCLUSIONS: If able to overcome participation challenges, lifestyle change programs in U.S. health systems may improve health among the growing patient population with diabetes.

Reduced Cardiovascular Disease Incidence With a National Lifestyle Change Program.

INTRODUCTION: Lifestyle change programs implemented within healthcare systems could reach many Americans, but their impact on cardiovascular disease (CVD) remains unclear. The MOVE! program is the largest lifestyle change program implemented in a healthcare setting in the U.S. This study aimed to determine whether MOVE! participation was associated with reduced CVD incidence. METHODS: This retrospective cohort study, analyzed in 2013-2015, used national Veterans Health Administration databases to identify MOVE! participants and eligible non-participants for comparison (2005-2012). Patients eligible for MOVE!-obese or overweight with a weight-related health condition, and no baseline CVD-were examined (N=1,463,003). Of these, 169,248 (12%) were MOVE! PARTICIPANTS: Patients were 92% male, 76% white, with mean age 52 years and BMI of 32. The main outcome was incidence of CVD (ICD-9 and procedure codes for coronary artery disease, cerebrovascular disease, peripheral vascular disease, and heart failure). RESULTS: Adjusting for age, race, sex, BMI, statin use, and baseline comorbidities, over a mean 4.9 years of follow-up, MOVE! participation was associated with lower incidence of total CVD (hazard ratio [HR]=0.83, 95% CI=0.80, 0.86); coronary artery disease (HR=0.81, 95% CI=0.77, 0.86); cerebrovascular disease (HR=0.87, 95% CI=0.82, 0.92); peripheral vascular disease (HR=0.89, 95% CI=0.83, 0.94); and heart failure (HR=0.78, 95% CI=0.74, 0.83). The association between MOVE! participation and CVD incidence remained significant when examined across categories of race/ethnicity, BMI, diabetes, hypertension, smoking status, and statin use. CONCLUSIONS: Although participation was limited, MOVE! was associated with reduced CVD incidence in a nationwide healthcare setting.

Translating What Works: A New Approach to Improve Diabetes Management.

BACKGROUND: The most efficacious strategies to improve diabetes control include case management, health care team changes, patient education, and facilitated transmission of patient data to clinicians ("facilitated relay"), but these strategies have not been translated to permit general use in clinical practice. METHODS: A web-based decision support program was developed to include these features, and assessed in patients who had A1c ≥7.0% despite using metformin with/without sulfonylureas or insulin. Staff entered patients' glucose data, obtained management recommendations, reviewed the plan with a clinician, and discussed the new plan with patients. RESULTS: 113 subjects were 96% male and 32% black, with average age 65.6 years and BMI 32.8. During prior primary care, A1c averaged 8.32 ± 0.16% (SEM). In all patients, baseline A1c was 8.18 ± 0.11%, and decreased to 7.54 ± 0.12%, 7.16 ± 0.13%, and 7.54 ± 0.16% at 3, 6, and 12 months, respectively, all P < .001. In 42 subjects who provided glucose data and made requested changes in medications, A1c was 8.12 ± 0.09% at baseline and fell to 7.29 ± 0.11%, 6.98 ± 0.10%, and 7.05 ± 0.10% at 3, 6, and 12 months, respectively, all P < .001. Chart review of 16 subjects followed for 12 months demonstrated that hypoglycemia (symptoms and/or glucose <70 mg/dl) averaged less than 1 episode/patient/month, and there was no severe hypoglycemia. CONCLUSIONS: A novel decision support program improved A1c with little hypoglycemia. Use of this approach should allow primary care teams to keep patients well controlled, and reduce the need for specialist referrals.

Weight loss and incidence of diabetes with the Veterans Health Administration MOVE! lifestyle change programme: an observational study.

BACKGROUND: Programmes for lifestyle change are aimed at improving health but little is known about their effectiveness in clinical settings. The Veterans Health Administration (VA) MOVE! lifestyle change programme is the largest in the USA. We investigated whether participation in MOVE! is associated with reduced incidence of diabetes. METHODS: We did a retrospective observational analysis of data from VA databases in overweight patients and obese patients with a weight-related disorder who had undergone at least 3 years of continuous outpatient care in 2005-12. We used generalised estimating equations to assess characteristics associated with MOVE! participation, and Cox's proportional hazards regression to analyse the association between participation and diabetes incidence. FINDINGS: Of 1·8 million eligible individuals, 238 540 (13%) participated in the MOVE! programme. 19 367 (1% overall, 8% of participants) met criteria for intense and sustained participation (at least eight sessions within 6 months over at least a 4-month span), which was associated with greater weight loss at 3 years than low-intensity or no participation (-2·2% vs -0·64% or 0·46%). Compared with non-participation, incidence of diabetes was reduced by intense and sustained participation (hazard ratio 0·67, 95% CI 0·61-0·74) and low-intensity participation (0·80, 0·77-0·83) in MOVE!. These patterns were consistent across sex, ethnic origin, and age. Participation was most beneficial in patients with high BMI or high random glucose concentrations at baseline (both pinteraction<0·0001). INTERPRETATION: Participation in the MOVE! programme was associated with weight loss and reduced incidence of diabetes, but the rate of participation was low and, therefore, selection bias could have exaggerated these effects. FUNDING: US Department of Veterans Affairs, National Institutes of Health.

Increased cardiovascular disease, resource use, and costs before the clinical diagnosis of diabetes in veterans in the southeastern U.S.

IMPORTANCE: Screening for diabetes might be more widespread if adverse associations with cardiovascular disease (CVD), resource use, and costs were known to occur earlier than conventional clinical diagnosis. OBJECTIVE: The purpose of this study was to determine whether adverse effects associated with diabetes begin prior to clinical diagnosis. DESIGN: Veterans with diabetes were matched 1:2 with controls by follow-up, age, race/ethnicity, gender, and VA facility. CVD was obtained from ICD-9 codes, and resource use and costs from VA datasets. SETTING: VA facilities in SC, GA, and AL. PARTICIPANTS: Patients with and without diagnosed diabetes. MAIN OUTCOME MEASURES: Diagnosed CVD, resource use, and costs. RESULTS: In this study, the 2,062 diabetic patients and 4,124 controls were 63 years old on average, 99 % male, and 29 % black; BMI was 30.8 in diabetic patients vs. 27.8 in controls (p<0.001). CVD prevalence was higher and there were more outpatient visits in Year -4 before diagnosis through Year +4 after diagnosis among diabetic vs. control patients (all p<0.01); in Year -2, CVD prevalence was 31 % vs. 24 %, and outpatient visits were 22 vs. 19 per year, respectively. Total VA costs/year/veteran were higher in diabetic than control patients from Year -4 ($4,083 vs. $2,754) through Year +5 ($8,347 vs. $5,700) (p<0.003) for each, reflecting underlying increases in outpatient, inpatient, and pharmacy costs (p<0.05 for each). Regression analysis showed that diabetes contributed an average of $1,748/year to costs, independent of CVD (p<0.001). CONCLUSIONS AND RELEVANCE: VA costs per veteran are higher--over $1,000/year before and $2,000/year after diagnosis of diabetes--due to underlying increases in outpatient, inpatient, and pharmacy costs, greater number of outpatient visits, and increased CVD. Moreover, adverse associations with veterans' health and the VA healthcare system occur early in the natural history of the disease, several years before diabetes is diagnosed. Since adverse associations begin before diabetes is recognized, greater consideration should be given to systematic screening in order to permit earlier detection and initiation of preventive management. Keeping frequency of CVD and marginal costs in line with those of patients before diabetes is currently diagnosed has the potential to save up to $2 billion a year.

Suicide attempts and completions in the emergency department in Veterans Affairs Hospitals.

BACKGROUND: This is the first study of suicide attempts and completions in the emergency department (ED) in a large national medical system. METHODS: All root cause analysis (RCA) reports completed of suicides and suicide attempts that occurred in ED in the Veterans Health Administration between 1 December 1999 and 31 December 2009 were reviewed. The method, location, anchor point for hanging and implement for cutting as well as the root causes were categorised. RESULTS: Ten per cent of all RCA reports of suicides and suicide attempts that occur within the hospital occur in the ED. Hanging, cutting and strangulation were the most common methods. The most common anchor point for hanging was doors, and the most common implement for cutting was a razor blade. In eight of the 10 cases of cutting, the implement was brought into the ED. The most common root causes were problems communicating risk and being short-staffed. CONCLUSIONS: Based on these results the following recommendations are made for helping to reduce suicide attempts in the ED: (1) use a systematic protocol and checklist to review mental health holding areas periodically in the ED for suicidal hazards; (2) develop and implement specialised protocols for suicidal patients that include continuous observation when possible; (3) conduct thorough contraband searches with suicidal patients; (4) designate specialised holding areas, when practically possible, for suicidal patients that are free of anchor points for hanging, sharps and medications, and medical equipment; and are isolated from exits to reduce the risk of elopement.

Pilot study evaluating a practice-based learning and improvement curriculum focusing on the development of system-level quality improvement skills.

BACKGROUND: We developed a practice-based learning and improvement (PBLI) curriculum to address important gaps in components of content and experiential learning activities through didactics and participation in systems-level quality improvement projects that focus on making changes in health care processes. METHODS: We evaluated the impact of our curriculum on resident PBLI knowledge, self-efficacy, and application skills. A quasi-experimental design assessed the impact of a curriculum (PBLI quality improvement systems compared with non-PBLI) on internal medicine residents' learning during a 4-week ambulatory block. We measured application skills, self-efficacy, and knowledge by using the Systems Quality Improvement Training and Assessment Tool. Exit evaluations assessed time invested and experiences related to the team projects and suggestions for improving the curriculum. RESULTS: The 2 groups showed differences in change scores. Relative to the comparison group, residents in the PBLI curriculum demonstrated a significant increase in the belief about their ability to implement a continuous quality improvement project (P  =  .020), comfort level in developing data collection plans (P  =  .010), and total knowledge scores (P < .001), after adjusting for prior PBLI experience. Participants in the PBLI curriculum also demonstrated significant improvement in providing a more complete aim statement for a proposed project after adjusting for prior PBLI experience (P  =  .001). Exit evaluations were completed by 96% of PBLI curriculum participants who reported high satisfaction with team performance. CONCLUSION: Residents in our curriculum showed gains in areas fundamental for PBLI competency. The observed improvements were related to fundamental quality improvement knowledge, with limited gain in application skills. This suggests that while heading in the right direction, we need to conceptualize and structure PBLI training in a way that integrates it throughout the residency program and fosters the application of this knowledge and these skills.

Development and preliminary evaluation of a practice-based learning and improvement tool for assessing resident competence and guiding curriculum development.

BACKGROUND: Although practice-based learning and improvement (PBLI) is now recognized as a fundamental and necessary skill set, we are still in need of tools that yield specific information about gaps in knowledge and application to help nurture the development of quality improvement (QI) skills in physicians in a proficient and proactive manner. We developed a questionnaire and coding system as an assessment tool to evaluate and provide feedback regarding PBLI self-efficacy, knowledge, and application skills for residency programs and related professional requirements. METHODS: Five nationally recognized QI experts/leaders reviewed and completed our questionnaire. Through an iterative process, a coding system based on identifying key variables needed for ideal responses was developed to score project proposals. The coding system comprised 14 variables related to the QI projects, and an additional 30 variables related to the core knowledge concepts related to PBLI. A total of 86 residents completed the questionnaire, and 2 raters coded their open-ended responses. Interrater reliability was assessed by percentage agreement and Cohen κ for individual variables and Lin concordance correlation for total scores for knowledge and application. Discriminative validity (t test to compare known groups) and coefficient of reproducibility as an indicator of construct validity (item difficulty hierarchy) were also assessed. RESULTS: Interrater reliability estimates were good (percentage of agreements, above 90%; κ, above 0.4 for most variables; concordances for total scores were R  =  .88 for knowledge and R  =  .98 for application). CONCLUSION: Despite the residents' limited range of experiences in the group with prior PBLI exposure, our tool met our goal of differentiating between the 2 groups in our preliminary analyses. Correcting for chance agreement identified some variables that are potentially problematic. Although additional evaluation is needed, our tool may prove helpful and provide detailed information about trainees' progress and the curriculum.

Comparison of extent of use, information accuracy, and functions for manual and electronic patient status boards.

PURPOSE: Electronic software packages to support patient tracking and disposition decision making in emergency departments (EDs) are being considered for implementation in many hospitals. We compared extent of use, information accuracy, and functions of manual and electronic patient status boards at 2 EDs where both were continuously in use. METHODS: Ethnographic observations were conducted at 2 Veterans Affairs Medical Center Emergency Departments using both manual and electronic patient status boards (100 h, 9 physicians at Site 1; 64 h, 14 physicians at Site 2). Data included board information collected at 20-min intervals, observable behavior while using boards, and interviews. RESULTS: Few physicians (3/9 [33%] Site 1; 0/14 [0%] Site 2) used the e-board, whereas all physicians used the whiteboards. Whiteboards had fewer inaccuracies (6/462 [1%] Site 1; 21/864 [3%] Site 2) than e-boards (62/462 [13%] Site 1; 107/864 [12%] Site 2). The primary functions of the whiteboard were to track real-time changes to patient identifiers, locations, nursing assignments, and pending activities; facilitate patient handoffs; inform physicians and nurses about newly arrived patients assigned to them; inform nurses of physicians' orders; and inform physicians of the status of ordered items. The primary functions of the e-board were to support electronic data entry (by clerks) of patient admitting and departure times; and highlight patients who had been in the ED for longer than 6 h. CONCLUSIONS: Whiteboards were more extensively used and had greater information accuracy than e-boards. Nevertheless, e-boards provided functionality not easily achievable with whiteboards.