"Mm-hm," "Uh-uh": are non-lexical conversational sounds deal breakers for the ambient clinical documentation technology?

OBJECTIVES: Ambient clinical documentation technology uses automatic speech recognition (ASR) and natural language processing (NLP) to turn patient-clinician conversations into clinical documentation. It is a promising approach to reducing clinician burden and improving documentation quality. However, the performance of current-generation ASR remains inadequately validated. In this study, we investigated the impact of non-lexical conversational sounds (NLCS) on ASR performance. NLCS, such as Mm-hm and Uh-uh, are commonly used to convey important information in clinical conversations, for example, Mm-hm as a "yes" response from the patient to the clinician question "are you allergic to antibiotics?" MATERIALS AND METHODS: In this study, we evaluated 2 contemporary ASR engines, Google Speech-to-Text Clinical Conversation ("Google ASR"), and Amazon Transcribe Medical ("Amazon ASR"), both of which have their language models specifically tailored to clinical conversations. The empirical data used were from 36 primary care encounters. We conducted a series of quantitative and qualitative analyses to examine the word error rate (WER) and the potential impact of misrecognized NLCS on the quality of clinical documentation. RESULTS: Out of a total of 135 647 spoken words contained in the evaluation data, 3284 (2.4%) were NLCS. Among these NLCS, 76 (0.06% of total words, 2.3% of all NLCS) were used to convey clinically relevant information. The overall WER, of all spoken words, was 11.8% for Google ASR and 12.8% for Amazon ASR. However, both ASR engines demonstrated poor performance in recognizing NLCS: the WERs across frequently used NLCS were 40.8% (Google) and 57.2% (Amazon), respectively; and among the NLCS that conveyed clinically relevant information, 94.7% and 98.7%, respectively. DISCUSSION AND CONCLUSION: Current ASR solutions are not capable of properly recognizing NLCS, particularly those that convey clinically relevant information. Although the volume of NLCS in our evaluation data was very small (2.4% of the total corpus; and for NLCS that conveyed clinically relevant information: 0.06%), incorrect recognition of them could result in inaccuracies in clinical documentation and introduce new patient safety risks.

Automatic speech recognition performance for digital scribes: a performance comparison between general-purpose and specialized models tuned for patient-clinician conversations.

One promising solution to address physician data entry needs is through the development of so-called "digital scribes," or tools which aim to automate clinical documentation via automatic speech recognition (ASR) of patient-clinician conversations. Evaluation of specialized ASR models in this domain, useful for understanding feasibility and development opportunities, has been difficult because most models have been under development. Following the commercial release of such models, we report an independent evaluation of four models, two general-purpose, and two for medical conversation with a corpus of 36 primary care conversations. We identify word error rates (WER) of 8.8%-10.5% and word-level diarization error rates (WDER) ranging from 1.8%-13.9%, which are generally lower than previous reports. The findings indicate that, while there is room for improvement, the performance of these specialized models, at least under ideal recording conditions, may be amenable to the development of downstream applications which rely on ASR of patient-clinician conversations.

An interview study with medical scribes on how their work may alleviate clinician burnout through delegated health IT tasks.

OBJECTIVES: To understand how medical scribes' work may contribute to alleviating clinician burnout attributable directly or indirectly to the use of health IT. MATERIALS AND METHODS: Qualitative analysis of semistructured interviews with 32 participants who had scribing experience in a variety of clinical settings. RESULTS: We identified 7 categories of clinical tasks that clinicians commonly choose to offload to medical scribes, many of which involve delegated use of health IT. These range from notes-taking and computerized data entry to foraging, assembling, and tracking information scattered across multiple clinical information systems. Some common characteristics shared among these tasks include: (1) time-consuming to perform; (2) difficult to remember or keep track of; (3) disruptive to clinical workflow, clinicians' cognitive processes, or patient-provider interactions; (4) perceived to be low-skill "clerical" work; and (5) deemed as adding no value to direct patient care. DISCUSSION: The fact that clinicians opt to "outsource" certain clinical tasks to medical scribes is a strong indication that performing these tasks is not perceived to be the best use of their time. Given that a vast majority of healthcare practices in the US do not have the luxury of affording medical scribes, the burden would inevitably fall onto clinicians' shoulders, which could be a major source for clinician burnout. CONCLUSIONS: Medical scribes help to offload a substantial amount of burden from clinicians-particularly with tasks that involve onerous interactions with health IT. Developing a better understanding of medical scribes' work provides useful insights into the sources of clinician burnout and potential solutions to it.

A scoping review of qualitative research in JAMIA: past contributions and opportunities for future work.

OBJECTIVE: Qualitative methods are particularly well-suited to studying the complexities and contingencies that emerge in the development, preparation, and implementation of technological interventions in real-world clinical practice, and much remains to be done to use these methods to their full advantage. We aimed to analyze how qualitative methods have been used in health informatics research, focusing on objectives, populations studied, data collection, analysis methods, and fields of analytical origin. METHODS: We conducted a scoping review of original, qualitative empirical research in JAMIA from its inception in 1994 to 2019. We queried PubMed to identify relevant articles, ultimately including and extracting data from 158 articles. RESULTS: The proportion of qualitative studies increased over time, constituting 4.2% of articles published in JAMIA overall. Studies overwhelmingly used interviews, observations, grounded theory, and thematic analysis. These articles used qualitative methods to analyze health informatics systems before, after, and separate from deployment. Providers have typically been the main focus of studies, but there has been an upward trend of articles focusing on healthcare consumers. DISCUSSION: While there has been a rich tradition of qualitative inquiry in JAMIA, its scope has been limited when compared with the range of qualitative methods used in other technology-oriented fields, such as human-computer interaction, computer-supported cooperative work, and science and technology studies. CONCLUSION: We recommend increased public funding for and adoption of a broader variety of qualitative methods by scholars, practitioners, and policy makers and an expansion of the variety of participants studied. This should lead to systems that are more responsive to practical needs, improving usability, safety, and outcomes.

How does medical scribes' work inform development of speech-based clinical documentation technologies? A systematic review.

OBJECTIVE: Use of medical scribes reduces clinician burnout by sharing the burden of clinical documentation. However, medical scribes are cost-prohibitive for most settings, prompting a growing interest in developing ambient, speech-based technologies capable of automatically generating clinical documentation based on patient-provider conversation. Through a systematic review, we aimed to develop a thorough understanding of the work performed by medical scribes in order to inform the design of such technologies. MATERIALS AND METHODS: Relevant articles retrieved by searching in multiple literature databases. We conducted the screening process following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) in guidelines, and then analyzed the data using qualitative methods to identify recurring themes. RESULTS: The literature search returned 854 results, 65 of which met the inclusion criteria. We found that there is significant variation in scribe expectations and responsibilities across healthcare organizations; scribes also frequently adapt their work based on the provider's style and preferences. Further, scribes' job extends far beyond capturing conversation in the exam room; they also actively interact with patients and the care team and integrate data from other sources such as prior charts and lab test results. DISCUSSION: The results of this study provide several implications for designing technologies that can generate clinical documentation based on naturalistic conversations taking place in the exam room. First, a one-size-fits-all solution will be unlikely to work because of the significant variation in scribe work. Second, technology designers need to be aware of the limited role that their solution can fulfill. Third, to produce comprehensive clinical documentation, such technologies will likely have to incorporate information beyond the exam room conversation. Finally, issues of patient consent and privacy have yet to be adequately addressed, which could become paramount barriers to implementing such technologies in realistic clinical settings. CONCLUSIONS: Medical scribes perform complex and delicate work. Further research is needed to better understand their roles in a clinical setting in order to inform the development of speech-based clinical documentation technologies.

Exercise and Repeated Testing Improves Accuracy of Laser Doppler Assessment of Microvascular Function Following Shortened (1-minute) Blood Flow Occlusion.

OBJECTIVES: To determine whether stability/accuracy of post-occlusive LDF following shortened, one-minute blood flow occlusion, increases in the post-exercise state or by averaging multiple measurements. METHODS: Six healthy adults (3F) underwent LDF eight times at rest and following exercise, assessing post-occlusive (one-minute occlusion) reactive hyperemia in the cutaneous microcirculation of the forefinger. Measured variables included: pre- and post-occlusion steady-state perfusion (Plat1, Plat2), maximum post-occlusive perfusion (Max), PkT, and the ratio Max/Plat1. RESULTS: Stability/accuracy of all variables improved performing measurements after exercise (p < 0.05 Plat 1, Plat 2, Max and Max/Plat1). PkT and Max/Plat 1 displayed the greatest accuracy at rest (26.6 ± 5.1% and 26.6 ± 4.4% average difference, %Diff, of single measurements from individual "true" means, respectively); for these variables, %Diff improved to 19.5 ± 5.3 and 17.6 ± 2.1, respectively, following exercise. Overall, averaging multiple measurements performed at rest also improved stability/accuracy in all variables. This improvement was comparable to that obtained with a single measurement following exercise. CONCLUSIONS: A standardized exercise stimulus prior to testing significantly improves stability/accuracy of LDF following shortened, one-minute blood flow occlusion. Our results suggest the possibilities of broader applications of exercise to optimize measurements from a variety of skin perfusion methodologies.

Exercise in pediatric type 1 diabetes.

The beneficial effects of exercise, including reduction of cardiovascular risk, are especially important in children with type 1 diabetes (T1DM), in whom incidence of lifetime cardiovascular complications remains elevated despite good glycemic control. Being able to exercise safely is therefore a paramount concern. Dysregulated metabolism in T1DM however, causes frequent occurrence of both hypo- and hyperglycemia, the former typically associated with prolonged, moderate exercise, the latter with higher intensity, if shorter, challenges. While very few absolute contraindications to exercising exist in these children, exercise should not be started with glycemia outside the 80-250 mg/dl range. Within this glycemic range, careful adjustments in insulin administration (reduction or infusion rate via insulin pumps, or overall reduction of dosage of multiple injections) should be combined with carbohydrate ingestion before/during exercise, based on prior, individual experience with specific exercise formats. Unfamiliar exercise should always be tackled with exceeding caution, based on known responses to other exercise formats. Finally, gaining a deep understanding of other complex exercise responses, such as the modulation of inflammatory status, which is a major determinant of the cardio-protective effects of exercise, can help determine which exercise formats and which individual metabolic conditions can lead to maximally beneficial health effects.

Altered insulin response to an acute bout of exercise in pediatric obesity.

Pediatric obesity typically induces insulin resistance, often later evolving into type 2 diabetes. While exercise, enhancing insulin sensitivity, is broadly used to prevent this transition, it is unknown whether alterations in the exercise insulin response pattern occur in obese children. Therefore, we measured exercise insulin responses in 57 healthy weight (NW), 20 overweight (OW), and 56 obese (Ob) children. Blood samples were drawn before and after 30 min of intermittent (2 min on, 1 min off) cycling at ~80% VO2max. In a smaller group (14 NW, 6 OW, 15 Ob), a high-fat meal was ingested 45 min preexercise. Baseline glycemia was similar and increased slightly and similarly in all groups during exercise. Basal insulin (pmol/L) was significantly higher in Ob vs. other groups; postexercise, insulin increased in NW (+7± 3) and OW (+5 ± 8), but decreased in Ob (-15±5, p < .0167 vs. NW). This insulin drop in Ob was disproportionately more pronounced in the half of Ob children with higher basal insulin (Ob-H). In all groups, high-fat feeding caused a rapid rise in insulin, promptly corrected by exercise. In Ob, however, insulin rose again 30 min postexercise. Our data indicates a distinct pattern of exercise-induced insulin modulation in pediatric obesity, possibly modulated by basal insulin concentrations.