RESUMO
Background: Large language models (LLMs) can assist providers in drafting responses to patient inquiries. We examined a prompt engineering strategy to draft responses for providers in the electronic health record. The aim was to evaluate the change in usability after prompt engineering. Materials and Methods: A pre-post study over 8 months was conducted across 27 providers. The primary outcome was the provider use of LLM-generated messages from Generative Pre-Trained Transformer 4 (GPT-4) in a mixed-effects model, and the secondary outcome was provider sentiment analysis. Results: Of the 7605 messages generated, 17.5% (n = 1327) were used. There was a reduction in negative sentiment with an odds ratio of 0.43 (95% CI, 0.36-0.52), but message use decreased (P < .01). The addition of nurses after the study period led to an increase in message use to 35.8% (P < .01). Discussion: The improvement in sentiment with prompt engineering suggests better content quality, but the initial decrease in usage highlights the need for integration with human factors design. Conclusion: Future studies should explore strategies for optimizing the integration of LLMs into the provider workflow to maximize both usability and effectiveness.
RESUMO
Objective: To assess for differences in patient care outcomes in the primary care setting for patients assigned to an independent practice panel (IPP) or a shared practice panel (SPP). Patients and Methods: We retrospectively reviewed the electronic health records of patients of 2 Mayo Clinic family medicine primary care clinics from January 1, 2019 to December 31, 2019. Patients were assigned to either an IPP (physician or advanced practice provider [APP]) or an SPP (physician and ≥1 APP). We assessed 6 measures of quality care and compared them between IPP and SPP groups: diabetes optimal care, hypertension control, depression remission at 6 months, breast cancer screening, cervical cancer screening, and colon cancer screening. Results: The study included 114,438 patients assigned to 140 family medicine panels during the study period: 87 IPPs and 53 SPPs. The IPP clinicians showed improved quality metrics compared with the SPP clinicians for the percentage of assigned patients achieving depression remission (16.6% vs 11.1%; P<.01). The SPP clinicians showed improved quality metrics compared with that of the IPP clinicians for the percentage of patients with cervical cancer screening (79.1% vs 74.2%; P<.01). The mean percentage of the panels achieving optimal diabetes control, hypertension control, colon cancer screening, and breast cancer screening were not significantly different between IPP and SPP panels. Conclusion: This study shows a considerable improvement in depression remission among IPP panels and in cervical cancer screening rates among SPP panels. This information may help to inform primary care team configuration.
RESUMO
COVID-19 vaccines are effective, but breakthrough infections have been increasingly reported. We conducted a test-negative case-control study to assess the durability of protection after full vaccination with BNT162b2 against polymerase chain reaction (PCR)-confirmed symptomatic SARS-CoV-2 infection, in a national medical practice from January 2021 through January 2022. We fit conditional logistic regression (CLR) models stratified on residential county and calendar time of testing to assess the association between time elapsed since vaccination and the odds of symptomatic infection or non-COVID-19 hospitalization (negative control), adjusted for several covariates. There were 5,985 symptomatic individuals with a positive test after full vaccination with BNT162b2 (cases) and 32,728 negative tests contributed by 27,753 symptomatic individuals after full vaccination (controls). The adjusted odds of symptomatic infection were higher 250 days after full vaccination versus at the date of full vaccination (Odds Ratio [OR]: 3.62, 95% CI: 2.52 to 5.20). The odds of infection were still lower 285 days after the first BNT162b2 dose as compared to 4 days after the first dose (OR: 0.50, 95% CI: 0.37 to 0.67), when immune protection approximates the unvaccinated status. Low rates of COVID-19 associated hospitalization or death in this cohort precluded analyses of these severe outcomes. The odds of non-COVID-19 associated hospitalization (negative control) decreased with time since vaccination, suggesting a possible underestimation of waning protection by this approach due to confounding factors. In summary, BNT162b2 strongly protected against symptomatic SARS-CoV-2 infection for at least 8 months after full vaccination, but the degree of protection waned significantly over this period.
RESUMO
OBJECTIVE: After a new electronic health record (EHR) was implemented at Mayo Clinic, a training program called reBoot Camp was created to enhance ongoing education in response to needs identified by physician leaders. MATERIALS AND METHODS: A reBoot camp focused on EHR topics pertinent to ambulatory care was offered from April 2018 through June 2020. There were 37 2-day sessions and 43 1-day sessions, with 673 unique participants. To evaluate outcomes of the reBoot camp, we used survey data to study baseline, immediate, and long-term perceptions of program satisfaction and self-assessed skills with the EHR. The study was conducted among practitioners at a large ambulatory practice network based in several states. Data were collected from April 2018 through January 2021. We analyzed automatically collected metadata and scores that evaluated the amount of personalization and proficiency of use. RESULTS: Confidence in skills increased by 13.5 points for general EHR use and was significant in 5 subdomains of use (13-18 point improvement). This degree of user confidence was maintained at the 6-month reassessment. The outcomes of configuration and proficiency scores also improved significantly. DISCUSSION: Ongoing education regarding EHR tools is necessary to support continued use of technology. This study was novel because of the amount and breadth of data collected, diversity of user participation, and validation that improvements were maintained over time. CONCLUSIONS: Participating in a reBoot camp significantly improved user confidence in each domain of the EHR and demonstrated use of best-practice tools. Users maintained gains at the 6-month evaluation phase.
Assuntos
Instituições de Assistência Ambulatorial , Registros Eletrônicos de Saúde , HumanosRESUMO
Importance: Recent reports on waning of COVID-19 vaccine-induced immunity have led to the approval and rollout of additional doses and booster vaccinations. Individuals at increased risk of SARS-CoV-2 infection are receiving additional vaccine doses in addition to the regimen that was tested in clinical trials. Risks and adverse event profiles associated with additional vaccine doses are currently not well understood. Objective: To evaluate the safety of third-dose vaccination with US Food and Drug Administration (FDA)-approved COVID-19 mRNA vaccines. Design, Setting, and Participants: This cohort study was conducted using electronic health record (EHR) data from December 2020 to October 2021 from the multistate Mayo Clinic Enterprise. Participants included all 47â¯999 individuals receiving 3-dose COVID-19 mRNA vaccines within the study setting who met study inclusion criteria. Participants were divided into 2 cohorts by vaccine brand administered and served as their own control groups, with no comparison made between cohorts. Data were analyzed from September through November 2021. Exposures: Three doses of an FDA-authorized COVID-19 mRNA vaccine, BNT162b2 or mRNA-1273. Main Outcomes and Measures: Vaccine-associated adverse events were assessed via EHR report. Adverse event risk was quantified using the percentage of study participants who reported the adverse event within 14 days after each vaccine dose and during a 14-day control period, immediately preceding the first vaccine dose. Results: Among 47â¯999 individuals who received 3-dose COVID-19 mRNA vaccines, 38â¯094 individuals (21â¯835 [57.3%] women; median [IQR] age, 67.4 [52.5-76.5] years) received BNT162b2 (79.4%) and 9905 individuals (5099 [51.5%] women; median [IQR] age, 67.7 [59.5-73.9] years) received mRNA-1273 (20.6%). Reporting of severe adverse events remained low after the third vaccine dose, with rates of pericarditis (0.01%; 95% CI, 0%-0.02%), anaphylaxis (0%; 95% CI, 0%-0.01%), myocarditis (0%; 95% CI, 0%-0.01%), and cerebral venous sinus thrombosis (no individuals) consistent with results from earlier studies. Significantly more individuals reported low-severity adverse events after the third dose compared with after the second dose, including fatigue (2360 individuals [4.92%] vs 1665 individuals [3.47%]; P < .001), lymphadenopathy (1387 individuals [2.89%] vs 995 individuals [2.07%]; P < .001), nausea (1259 individuals [2.62%] vs 979 individuals [2.04%]; P < .001), headache (1185 individuals [2.47%] vs 992 individuals [2.07%]; P < .001), arthralgia (1019 individuals [2.12%] vs 816 individuals [1.70%]; P < .001), myalgia (956 individuals [1.99%] vs 784 individuals [1.63%]; P < .001), diarrhea (817 individuals [1.70%] vs 595 individuals [1.24%]; P < .001), fever (533 individuals [1.11%] vs 391 individuals [0.81%]; P < .001), vomiting (528 individuals [1.10%] vs 385 individuals [0.80%]; P < .001), and chills (224 individuals [0.47%] vs 175 individuals [0.36%]; P = .01). Conclusions and Relevance: This study found that although third-dose vaccination against SARS-CoV-2 infection was associated with increased reporting of low-severity adverse events, risk of severe adverse events remained comparable with risk associated with the standard 2-dose regime. These findings suggest the safety of third vaccination doses in individuals who were eligible for booster vaccination at the time of this study.
Assuntos
Vacinas contra COVID-19 , COVID-19 , Idoso , Vacina BNT162 , COVID-19/prevenção & controle , Vacinas contra COVID-19/efeitos adversos , Estudos de Coortes , Registros Eletrônicos de Saúde , Feminino , Humanos , Masculino , RNA Mensageiro , SARS-CoV-2 , Vacinação/efeitos adversos , Vacinas Sintéticas , Vacinas de mRNARESUMO
BACKGROUND: mRNA coronavirus disease 2019 (COVID-19) vaccines are safe and effective, but increasing reports of breakthrough infections highlight the need to vigilantly monitor and compare the effectiveness of these vaccines. METHODS: We retrospectively compared protection against symptomatic infection conferred by mRNA-1273 and BNT162b2 at Mayo Clinic sites from December 2020 to September 2021. We used a test-negative case-control design to estimate vaccine effectiveness (VE) and to compare the odds of symptomatic infection after full vaccination with mRNA-1273 versus BNT162b2, while adjusting for age, sex, race, ethnicity, geography, comorbidities, and calendar time of vaccination and testing. FINDINGS: Both vaccines were highly effective over the study duration (VEmRNA-1273: 84.1%, 95% confidence interval [CI]: 81.6%-86.2%; VEBNT162b2: 75.6%, 95% CI: 72.2%-78.7%), but their effectiveness was reduced during July-September (VEmRNA-1273: 75.6%, 95% CI: 70.1%-80%; VEBNT162b2: 63.5%, 95% CI: 55.8%-69.9%) as compared to December-May (VEmRNA-1273: 93.7%, 95% CI: 90.4%-95.9%; VEBNT162b2: 85.7%, 95% CI: 81.4%-88.9%). Adjusted for demographic characteristics, clinical comorbidities, time of vaccination, and time of testing, the odds of experiencing a symptomatic breakthrough infection were lower after full vaccination with mRNA-1273 than with BNT162b2 (odds ratio: 0.60; 95% CI: 0.55-0.67). CONCLUSIONS: Both mRNA-1273 and BNT162b2 strongly protect against symptomatic severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection. It is imperative to continue monitoring and comparing available vaccines over time and with respect to emerging variants to inform public and global health decisions. FUNDING: This study was funded by nference.
Assuntos
COVID-19 , Vacina de mRNA-1273 contra 2019-nCoV , Vacina BNT162 , COVID-19/prevenção & controle , Vacinas contra COVID-19/uso terapêutico , Humanos , Estudos Retrospectivos , SARS-CoV-2/genéticaRESUMO
COVID-19 vaccines are effective, but breakthrough infections have been increasingly reported. We conducted a test-negative case-control study to assess the durability of protection against symptomatic infection after vaccination with mRNA-1273. We fit conditional logistic regression (CLR) models stratified on residential county and calendar date of SARS-CoV-2 PCR testing to assess the association between the time elapsed since vaccination and the odds of symptomatic infection, adjusted for several covariates. There were 2,364 symptomatic individuals who had a positive SARS-CoV-2 PCR test after full vaccination with mRNA-1273 ("cases") and 12,949 symptomatic individuals who contributed 15,087 negative tests after full vaccination ("controls"). The odds of symptomatic infection were significantly higher 250 days after full vaccination compared to the date of full vaccination (Odds Ratio [OR]: 2.47, 95% confidence interval [CI]: 1.19-5.13). The odds of non-COVID-19 associated hospitalization and non-COVID-19 pneumonia (negative control outcomes) remained relatively stable over the same time interval (Day 250 ORNon-COVID Hospitalization: 0.68, 95% CI: 0.47-1.0; Day 250 ORNon-COVID Pneumonia: 1.11, 95% CI: 0.24-5.2). The odds of symptomatic infection remained significantly lower almost 300 days after the first mRNA-1273 dose as compared to 4 days after the first dose, when immune protection approximates the unvaccinated state (OR: 0.26, 95% CI: 0.17-0.39). Low rates of COVID-19 associated hospitalization or death in this cohort precluded analyses of these severe outcomes. In summary, mRNA-1273 robustly protected against symptomatic SARS-CoV-2 infection at least 8 months after full vaccination, but the degree of protection waned over this time period.
RESUMO
OBJECTIVE: To report the Mayo Clinic experience with coronavirus disease 2019 (COVID-19) related to patient outcomes. METHODS: We conducted a retrospective chart review of patients with COVID-19 diagnosed between March 1, 2020, and July 31, 2020, at any of the Mayo Clinic sites. We abstracted pertinent comorbid conditions such as age, sex, body mass index, Charlson Comorbidity Index variables, and treatments received. Factors associated with hospitalization and mortality were assessed in univariate and multivariate models. RESULTS: A total of 7891 patients with confirmed COVID-19 infection with research authorization on file received care across the Mayo Clinic sites during the study period. Of these, 7217 patients were adults 18 years or older who were analyzed further. A total of 897 (11.4%) patients required hospitalization, and 354 (4.9%) received care in the intensive care unit (ICU). All hospitalized patients were reviewed by a COVID-19 Treatment Review Panel, and 77.5% (695 of 897) of inpatients received a COVID-19-directed therapy. Overall mortality was 1.2% (94 of 7891), with 7.1% (64 of 897) mortality in hospitalized patients and 11.3% (40 of 354) in patients requiring ICU care. CONCLUSION: Mayo Clinic outcomes of patients with COVID-19 infection in the ICU, hospital, and community compare favorably with those reported nationally. This likely reflects the impact of interprofessional multidisciplinary team evaluation, effective leveraging of clinical trials and available treatments, deployment of remote monitoring tools, and maintenance of adequate operating capacity to not require surge adjustments. These best practices can help guide other health care systems with the continuing response to the COVID-19 pandemic.