Measuring and Improving Evidence-Based Patient Care Using a Web-Based Gamified Approach in Primary Care (QualityIQ): Randomized Controlled Trial.

BACKGROUND: Unwarranted variability in clinical practice is a challenging problem in practice today, leading to poor outcomes for patients and low-value care for providers, payers, and patients. OBJECTIVE: In this study, we introduced a novel tool, QualityIQ, and determined the extent to which it helps primary care physicians to align care decisions with the latest best practices included in the Merit-Based Incentive Payment System (MIPS). METHODS: We developed the fully automated QualityIQ patient simulation platform with real-time evidence-based feedback and gamified peer benchmarking. Each case included workup, diagnosis, and management questions with explicit evidence-based scoring criteria. We recruited practicing primary care physicians across the United States into the study via the web and conducted a cross-sectional study of clinical decisions among a national sample of primary care physicians, randomized to continuing medical education (CME) and non-CME study arms. Physicians "cared" for 8 weekly cases that covered typical primary care scenarios. We measured participation rates, changes in quality scores (including MIPS scores), self-reported practice change, and physician satisfaction with the tool. The primary outcomes for this study were evidence-based care scores within each case, adherence to MIPS measures, and variation in clinical decision-making among the primary care providers caring for the same patient. RESULTS: We found strong, scalable engagement with the tool, with 75% of participants (61 non-CME and 59 CME) completing at least 6 of 8 total cases. We saw significant improvement in evidence-based clinical decisions across multiple conditions, such as diabetes (+8.3%, P<.001) and osteoarthritis (+7.6%, P=.003) and with MIPS-related quality measures, such as diabetes eye examinations (+22%, P<.001), depression screening (+11%, P<.001), and asthma medications (+33%, P<.001). Although the CME availability did not increase enrollment in the study, participants who were offered CME credits were more likely to complete at least 6 of the 8 cases. CONCLUSIONS: Although CME availability did not prove to be important, the short, clinically detailed case simulations with real-time feedback and gamified peer benchmarking did lead to significant improvements in evidence-based care decisions among all practicing physicians. TRIAL REGISTRATION: ClinicalTrials.gov NCT03800901; https://clinicaltrials.gov/ct2/show/NCT03800901.

Initiative to Improve Evidence-Based Chronic Obstructive Pulmonary Disease Hospitalist Care Using a Novel On-Line Gamification Patient Simulation Tool: A Prospective Study.

Chronic obstructive pulmonary disease (COPD) remains a leading cause of morbidity and mortality. Much of the disease burden comes from exacerbations requiring hospitalization. Unwarranted care variation and divergence from evidence-based COPD management guidelines among hospitalists is a leading driver of the poor outcomes and excess costs associated with COPD-related hospitalizations. We engaged with Novant Health hospitalists to determine if measurement and feedback using fixed-choice simulated patients improves evidence-based care delivery and reduces costs. We created a series of gamified acute-care COPD case simulations with real-time feedback over 16 weeks then performed a year-over-year analytic comparison of the cost, length of stay (LOS), and revisits over the six months prior to the introduction of the simulated patients, the four months while caring for the simulated patients, and the six months after. In total, 245 hospitalists from 15 facilities at Novant Health participated. At baseline, the overall quality-of-care was measured as 58.4% + 12.3%, with providers correctly identifying COPD exacerbation in 92.4% of cases but only identifying the grade and group in 61.9% and 49.5% of cases, respectively. By the study end, the quality-of-care had improved 10.5% (p < 0.001), including improvements in identifying the grade (+9.7%, p = 0.044) and group (+8.4%, p = 0.098). These improvements correlated with changes in real-world performance data, including a 19% reduction in COPD-related pharmacy costs. Overall, the annualized impact of COPD improvements led to 233 fewer inpatient days, 371 fewer revisit days, and inpatient savings totaling nearly $1 million. Engaging practicing providers with patient simulation-based serial measurements and gamified evidence-based feedback potentially reduces inpatient costs while simultaneously reducing patient LOS and revisit rates.

Finding the clinical utility of 1,5-anhydroglucitol among primary care practitioners.

BACKGROUND: HbA1c is widely used as the standard measure to track glycemic control in patients with diabetes and pre-diabetes but measures average levels of glycated hemoglobin over two to three months, with limited utility in the presence of recent and/or short-term fluctuations in glycemic control, which are correlated with worse patient outcomes. METHODS: We examined the clinical utility of 1-5-anhydroglucitol (1,5-AG) in six different, but common, case types of diabetes patients with short-term glycemic variability. We conducted a randomized controlled trial of simulated patients to examine the clinical practice patterns of primary care physicians before and after introducing 1,5-AG. The 145 participants were randomly assigned into standard care or standard care + 1,5-AG arms. Provider care was reviewed against explicit evidence-based care standards. RESULTS: At baseline, we saw no difference between the two study arms in clinical quality of care provided (p = 0.997). After introduction of 1,5-AG, standard care + 1,5-AG providers performed 3.2% better than controls (p = 0.025. In diagnosis and treatment, there was a slight, but nonsignificant trend toward better care (+1.1%, p = 0.507) for intervention providers. Upon disaggregation by case, almost all the improvement occurred in the medication-induced hyperglycemia patients (+8.1%, p = 0.047). CONCLUSIONS: A nationally representative sample of primary care physicians demonstrated that of six different cases used in this study, 1,5-AG was found to be most effective increasing awareness of poor glucose control in medication-induced hyperglycemia. If 1,5-AG is used in this particular circumstance, the overall savings to the healthcare system is estimated to be $28 million.

Reducing Unwarranted Oncology Care Variation Across a Clinically Integrated Network: A Collaborative Physician Engagement Strategy.

PURPOSE: Addressing unwarranted clinical variation in oncology is a high priority for health systems that aspire to ensure consistent levels of high-quality and cost-effective care. Efforts to improve clinical practice and standardize care have proven challenging. Advocate Physician Partners undertook a patient simulation-based practice measurement and feedback project that was focused on breast and lung cancer to engage oncologists in the care standardization process. METHODS: One hundred three medical oncologists cared for online simulated patients using the Clinical Performance and Value platform, receiving feedback on how their care decisions compared with evidence-based guidelines and their peers. We repeated this process every 4 months over six rounds, measuring changes in quality-of-care scores. We then compared simulated patient results with available patient-level claims data. RESULTS: Over the course of the project, overall quality-of-care scores improved 11.9% (P < .001). Diagnostic accuracy increased 6.7% (P < .001) and correlated with improved treatment scores, including a nearly 10-percentage point increase in evidence-based chemotherapy regimens (P = .009) and a 56% increase in addressing palliative needs for patients with late-stage disease (P < .001). Unnecessary test ordering declined 25% (P < .001). We compared these results with available patient data and observed concordance with the metastatic imaging workup order rate for early-stage breast cancer. As unnecessary workups declined in the simulations and became more closely aligned with evidence-based guidelines, we saw similar rates of decline in the patient-level data. CONCLUSION: This study demonstrates that an oncology care standardization system that combines simulated patients with serial feedback increases evidence-based and cost-effective clinical decisions in patient simulations and patient-level data.

Establishing Clinical Utility for Diagnostic Tests Using a Randomized Controlled, Virtual Patient Trial Design.

Demonstrating clinical utility for diagnostic tests and securing coverage and reimbursement requires high quality and, ideally, randomized controlled trial (RCT) data. Traditional RCTs are often too costly, slow, and cumbersome for diagnostic firms. Alternative data options are needed. We evaluated four RCTs using virtual patients to demonstrate clinical utility. Each study used a similar pre-post intervention, two round design to facilitate comparison. Representative samples of physicians were recruited and randomized into control and intervention arms. All physicians were asked to care for their virtual patients during two assessment rounds, separated by a multi-week time interval. Between rounds, intervention physicians reviewed educational materials on the diagnostic test. All physician responses were scored against evidence-based care criteria. RCTs using virtual patients can demonstrate clinical utility for a variety of diagnostic test types, including: (1) an advanced multi-biomarker blood test, (2) a chromosomal microarray, (3) a proteomic assay analysis, and (4) a multiplex immunofluorescence imaging platform. In two studies, utility was demonstrated for all targeted patient populations, while in the other two studies, utility was only demonstrated for a select sub-segment of the intended patient population. Of these four tests, two received positive coverage decisions from Palmetto, one utilized the study results to support commercial payer adjudications, and the fourth company went out of business. RCTs using virtual patients are a cost-effective approach to demonstrate the presence or absence of clinical utility.

Reducing Unneeded Clinical Variation in Sepsis and Heart Failure Care to Improve Outcomes and Reduce Cost: A Collaborative Engagement with Hospitalists in a MultiState System.

OBJECTIVE: To (1) measure hospitalist care for sepsis and heart failure patients using online simulated patients, (2) improve quality and reduce cost through customized feedback, and (3) compare patient-level outcomes between project participants and nonparticipants. METHODS: We conducted a prospective, quasi-controlled cohort study of hospitalists in eight hospitals matched with comparator hospitalists in six nonparticipating hospitals across the AdventHealth system. We provided measurement and feedback to participants using Clinical Performance and Value (CPV) vignettes to measure and track quality improvement. We then compared length of stay (LOS) and cost results between the two groups. RESULTS: 107 providers participated in the study. Over two years, participants improved CPV scores by nearly 8% (P < .001), with improvements in utilization of the three-hour sepsis bundle (46.0% vs 57.7%; P = .034) and ordering essential medical treatment elements for heart failure (58.2% vs 72.1%; P = .038). In study year one, average LOS observed/expected (O/E) rates dropped by 8% for participants, compared to 2.5% in the comparator group, equating to an additional 570 hospital days saved among project participants. In study year two, cost O/E rates improved from 1.16 to 0.98 for participants versus 1.14 to 1.01 in the comparator group. Based on these improvements, we calculated total cost savings of $6.2 million among study participants, with $3.8 million linked to system-wide improvements and an additional $2.4 million in savings attributable to this project. CONCLUSIONS: CPV case simulation-based measurement and feedback helped drive improvements in evidence-based care that translated into lower costs and LOS, above-and-beyond other improvements at AdventHealth.

A Better Pathway? Building Consensus and Engaging Providers with Feedback to Improve and Standardize Cancer Care.

INTRODUCTION: Unwanted clinical variation is common across the United States health care system and is particularly vexing in oncology owing to the complexity, morbidity, and high cost of the disease. Efforts to standardize care including guidelines and continuing medical education have had only limited impact. Disease-specific oncology clinical pathways hold the promise of reducing variation but have been hampered by a lack of ownership and accountability among oncology providers. MATERIALS AND METHODS: We describe the utility of combining a patient simulation-based clinical variation measurement with the in-house development of multidisciplinary breast cancer pathways at a National Cancer Institute-designated cancer center. RESULTS: At baseline, we found high variation in care decisions across the multidisciplinary team and within individual specialties in the management of simulated patients. Development and introduction of breast cancer clinical pathways combined with individual and group feedback on pathway adherence led to significant increases in pathway-aligned care decisions and decreases in measured variation. Overall quality scores increased from 47.5% to 61.1% (P < .001), with the largest improvement in diagnostic accuracy (+22.1%). Providers also ordered fewer unnecessary tests, saving an estimated $305 per patient case. Adherence to preferred chemotherapy regimens increased for both medical oncologists (+16%) and other members of the multidisciplinary team (+19%). CONCLUSION: Our work shows that a structured process to measure clinical variation and provide personalized feedback to an oncology multidisciplinary team drives adoption of evidence-based pathways, less unneeded spending, and higher quality care for patients.

Engaging Primary Care Providers to Reduce Unwanted Clinical Variation and Support ACO Cost and Quality Goals: A Unique Provider-Payer Collaboration.

This project was undertaken to reduce unneeded variation among practicing primary care clinicians participating in an accountable care organization (ACO) and to raise quality and reduce costs. This real-world, quasi-controlled experiment compared ACO target improvements between 3 participating geographic regions and members within the ProHealth ACO against nonparticipating regions and members. The authors used a novel care standardization initiative to engage participating providers. This was a 2-year longitudinal study with 6 rounds of serially measured provider care decisions and customized individual and group improvement feedback. Participating providers cared for online patient simulations as they would actual patients, and their care decisions were scored against evidence-based guidelines. This approach generated significant increases in evidence-based quality scores (+27%) and reductions in unneeded testing (-55%) in the patient simulations. Improvements in the online simulated patients correlated with improvements in patient-level ACO quality measures, which showed gains above and beyond the quasi-control group. Reductions calculated for spending on unneeded tests and specialist referrals exceeded $4.8 million. This study found that supporting practicing physicians in ACOs with evidence-based feedback significantly improved care and cost-efficiency.

Impact of a protein-based assay that predicts prostate cancer aggressiveness on urologists' recommendations for active treatment or active surveillance: a randomized clinical utility trial.

BACKGROUND: Of the more than 1.1 million men diagnosed worldwide annually with prostate cancer, the majority have indolent tumors. Distinguishing between aggressive and indolent cancer is an important clinical challenge. The current approaches for assessing tumor aggressiveness are recognized as insufficient. A validated protein-based assay has been shown to predict tumor aggressiveness from prostate biopsy. The main objective of this study was to measure the clinical utility of this new assay in the management of early-stage prostate cancer. METHODS: One hundred twenty nine board-certified urologists were asked to participate in a randomized, two-arm experiment. We collected data over 2 rounds using simulated clinical cases administered via an online platform. The cases were all newly diagnosed Gleason 3 + 3 or 3 + 4 prostate camcer patients. Urologists in the intervention arm received a 15-min webinar on this protein-based assay and given assay test results for their simulated patients in round 2. Each case had a preferred recommendation of either active surveillance or active treatment. The measured outcome was rate of preferred recommendation, defined as urologists who recommended the proper treatment course. Analyses were done using difference-in-difference estimations. RESULTS: Using multinomial logistical regression, urologists who were given the assay results were significantly more likely to choose the preferred recommendation (active surveillance or active treatment) compared to controls (p = 0.004). These urologists were also significantly more likely to involve their patients in the treatment decision compared to controls (p = 0.001). CONCLUSIONS: By providing additional information to inform the physician's treatment plan, a protein-based assay shows demonstrable clinical utility confirmed through a rigorous randomized controlled study design and regression analyses to test for effects.

Clinical Utility of a Comprehensive, Whole Genome CMA Testing Platform in Pediatrics: A Prospective Randomized Controlled Trial of Simulated Patients in Physician Practices.

BACKGROUND: Developmental disorders (DD), including autism spectrum disorder (ASD) and intellectual disability (ID), are a common group of clinical manifestations caused by a variety of genetic abnormalities. Genetic testing, including chromosomal microarray (CMA), plays an important role in diagnosing these conditions, but CMA can be limited by incomplete coverage of genetic abnormalities and lack of guidance for conditions rarely seen by treating physicians. METHODS: We conducted a longitudinal, randomized controlled trial investigating the impact of a higher resolution 2.8 million (MM) probe-CMA test on the quality of care delivered by practicing general pediatricians and specialists. To overcome the twin problems of finding an adequate sample size of multiple rare conditions and under/incorrect diagnoses, we used standardized simulated patients known as CPVs. Physicians, randomized into control and intervention groups, cared for the CPV pediatric patients with DD/ASD/ID. Care responses were scored against evidence-based criteria. In round one, participants could order diagnostic tests including existing CMA tests. In round two, intervention physicians could order the 2.8MM probe-CMA test. Outcome measures included overall quality of care and quality of the diagnosis and treatment plan. RESULTS: Physicians ordering CMA testing had 5.43% (p<0.001) higher overall quality scores than those who did not. Intervention physicians ordering the 2.8MM probe-CMA test had 7.20% (p<0.001) higher overall quality scores. Use of the 2.8MM probe-CMA test led to a 10.9% (p<0.001) improvement in the diagnosis and treatment score. Introduction of the 2.8MM probe-CMA test led to significant improvements in condition-specific interventions including an 8.3% (p = 0.04) improvement in evaluation and therapy for gross motor delays caused by Hunter syndrome, a 27.5% (p = 0.03) increase in early cognitive intervention for FOXG1-related disorder, and an 18.2% (p<0.001) improvement in referrals to child neurology for Dravet syndrome. CONCLUSION: Physician use of the 2.8MM probe-CMA test significantly improves overall quality as well as diagnosis and treatment quality for simulated cases of pediatric DD/ASD/ID patients, and delivers additional clinical utility over existing CMA tests.

Improving Clinical Practice Using a Novel Engagement Approach: Measurement, Benchmarking and Feedback, A Longitudinal Study.

BACKGROUND: Poor clinical outcomes are caused by multiple factors such as disease progression, patient behavior, and structural elements of care. One other important factor that affects outcome is the quality of care delivered by a provider at the bedside. Guidelines and pathways have been developed with the promise of advancing evidence-based practice. Yet, these alone have shown mixed results or fallen short in increasing adherence to quality of care. Thus, effective, novel tools are required for sustainable practice change and raising the quality of care. METHODS: The study focused on benchmarking and measuring variation and improving care quality for common types of breast cancer at four sites across the United States, using a set of 12 Clinical Performance and Value(®) (CPV(®)) vignettes per site. The vignettes simulated online cases that replicate a typical visit by a patient as the tool to engage breast cancer providers and to identify and assess variation in adherence to evidence-based practice guidelines and pathways. RESULTS: Following multiple rounds of CPV measurement, benchmarking and feedback, we found that scores had increased significantly between the baseline round and the final round (P < 0.001) overall and for all domains. By round 4 of the study, the overall score increased by 14% (P < 0.001), and the diagnosis with treatment plan domain had an increase of 12% (P < 0.001) versus baseline. CONCLUSION: We found that serially engaging breast cancer providers with a validated clinical practice engagement and measurement tool, the CPVs, markedly increased quality scores and adherence to clinical guidelines in the simulated patients. CPVs were able to measure differences in clinical skill improvement and detect how fast improvements were made.

Low Rates of Genetic Testing in Children With Developmental Delays, Intellectual Disability, and Autism Spectrum Disorders.

To explore the routine and effective use of genetic testing for patients with intellectual disability and developmental delay (ID/DD), we conducted a prospective, randomized observational study of 231 general pediatricians (40%) and specialists (60%), using simulated patients with 9 rare pediatric genetic illnesses. Participants cared for 3 randomly assigned simulated patients, and care responses were scored against explicit evidence-based criteria. Scores were calculated as a percentage of criteria completed. Care varied widely, with a median overall score of 44.7% and interquartile range of 36.6% to 53.7%. Diagnostic accuracy was low: 27.4% of physicians identified the correct primary diagnosis. Physicians ordered chromosomal microarray analysis in 55.7% of cases. Specific gene sequence testing was used in 1.4% to 30.3% of cases. This study demonstrates that genetic testing is underutilized, even for widely available tests. Further efforts to educate physicians on the clinical utility of genetic testing may improve diagnosis and care in these patients.

Six RNA viruses and forty-one hosts: viral small RNAs and modulation of small RNA repertoires in vertebrate and invertebrate systems.

We have used multiplexed high-throughput sequencing to characterize changes in small RNA populations that occur during viral infection in animal cells. Small RNA-based mechanisms such as RNA interference (RNAi) have been shown in plant and invertebrate systems to play a key role in host responses to viral infection. Although homologs of the key RNAi effector pathways are present in mammalian cells, and can launch an RNAi-mediated degradation of experimentally targeted mRNAs, any role for such responses in mammalian host-virus interactions remains to be characterized. Six different viruses were examined in 41 experimentally susceptible and resistant host systems. We identified virus-derived small RNAs (vsRNAs) from all six viruses, with total abundance varying from "vanishingly rare" (less than 0.1% of cellular small RNA) to highly abundant (comparable to abundant micro-RNAs "miRNAs"). In addition to the appearance of vsRNAs during infection, we saw a number of specific changes in host miRNA profiles. For several infection models investigated in more detail, the RNAi and Interferon pathways modulated the abundance of vsRNAs. We also found evidence for populations of vsRNAs that exist as duplexed siRNAs with zero to three nucleotide 3' overhangs. Using populations of cells carrying a Hepatitis C replicon, we observed strand-selective loading of siRNAs onto Argonaute complexes. These experiments define vsRNAs as one possible component of the interplay between animal viruses and their hosts.

Bypass suppression of small-plaque phenotypes by a mutation in poliovirus 2A that enhances apoptosis.

The rate of protein secretion in host cells is inhibited during infection with several different picornaviruses, with consequences likely to have significant effects on viral growth, spread, and pathogenesis. This Sin(+) (secretion inhibition) phenotype has been documented for poliovirus, foot-and-mouth disease virus, and coxsackievirus B3 and can lead to reduced cell surface expression of major histocompatibility complex class I and tumor necrosis factor receptor as well as reduced extracellular secretion of induced cytokines such as interleukin-6 (IL-6), IL-8, and beta interferon. The inhibition of protein secretion is global, affecting the movement of all tested cargo proteins through the cellular secretion apparatus. To test the physiological significance of the Sin(-) and Sin(+) phenotypes in animal models, Sin(-) mutant viruses are needed that fail to inhibit host protein secretion and also exhibit robust growth properties. To identify such Sin(-) mutant polioviruses, we devised a fluorescence-activated cell sorter-based screen to select virus-infected cells that nevertheless expressed newly synthesized surface proteins. After multiple rounds of selection, candidate Sin(-) mutant viruses were screened for genetic stability, increased secretion of cargo molecules and wild-type translation and growth properties. A newly identified Sin(-) mutant poliovirus that contained coding changes in nonstructural proteins 2A (N32D) and 2C (E253G) was characterized. In this virus, the 2C mutation is responsible for the Sin(-) phenotype and the 2A mutation suppresses a resulting growth defect by increasing the rate of cell death and therefore the rate of viral spread. The 2A-N32D suppressor mutation was not allele specific and, by increasing the rate of cellular apoptosis, affected a completely different pathway than the 2C-E253G Sin(-) mutation. Therefore, the 2A mutation suppresses the 2C-E253G mutant phenotype by a bypass suppression mechanism.

Role of microtubules in extracellular release of poliovirus.

Cellular autophagy, a process that directs cytosolic contents to the endosomal and lysosomal pathways via the formation of double-membraned vesicles, is a crucial aspect of innate immunity to many intracellular pathogens. However, evidence is accumulating that certain RNA viruses, such as poliovirus, subvert this pathway to facilitate viral growth. The autophagosome-like membranes induced during infection with wild-type poliovirus were found to be, unlike cellular autophagosomes, relatively immobile. Their mobility increased upon nocodazole treatment, arguing that vesicular tethering is microtubule dependent. In cells infected with a mutant virus that is defective in its interaction with the host cytoskeleton and secretory pathway, vesicle movement increased, indicating reduced tethering. In all cases, the release of tethering correlated with increased amounts of extracellular virus, which is consistent with the hypothesis that small amounts of cytosol and virus entrapped by double-membraned structures could be released via fusion with the plasma membrane. We propose that this extracellular delivery of cytoplasmic contents be termed autophagosome-mediated exit without lysis (AWOL). This pathway could explain the observed exit, in the apparent absence of cellular lysis, of other cytoplasmic macromolecular complexes, including infectious agents and complexes of aggregated proteins.

Characterization of the follicular dendritic cell reservoir of human immunodeficiency virus type 1.

Throughout the natural course of human immunodeficiency virus (HIV) infection, follicular dendritic cells (FDCs) trap and retain large quantities of particle-associated HIV RNA in the follicles of secondary lymphoid tissue. We have previously found that murine FDCs in vivo could maintain trapped virus particles in an infectious state for at least 9 months. Here we sought to determine whether human FDCs serve as an HIV reservoir, based on the criteria that virus therein must be replication competent, genetically diverse, and archival in nature. We tested our hypothesis using postmortem cells and tissues obtained from three HIV-infected subjects and antemortem blood samples obtained from one of these subjects. Replication competence was determined using coculture, while genetic diversity and the archival nature of virus were established using phylogenetic and population genetics methods. We found that FDC-trapped virus was replication competent and demonstrated greater genetic diversity than that of virus found in most other tissues and cells. Antiretrovirus-resistant variants that were not present elsewhere were also detected on FDCs. Furthermore, genetic similarity was observed between FDC-trapped HIV and viral species recovered from peripheral blood mononuclear cells obtained 21 and 22 months antemortem, but was not present in samples obtained 4 and 18 months prior to the patient's death, indicating that FDCs can archive HIV. These data indicate that FDCs represent a significant reservoir of infectious and diverse HIV, thereby providing a mechanism for viral persistence for months to years.