Clustering Research Proposal Submissions to Understand the Unmet Needs of Military Clinicians.

INTRODUCTION: The Advanced Medical Technology Initiative (AMTI) program solicits research proposals for technology demonstrations and performance improvement projects in the domain of military medicine. Advanced Medical Technology Initiative is managed by the U.S. Army Telemedicine and Advanced Technology Research Center (TATRC). Advanced Medical Technology Initiative proposals span a wide range of topics, for example, treatment of musculoskeletal injury, application of virtual health technology, and demonstration of medical robots. The variety and distribution of central topics in these proposals (problems to be solved and technological solutions proposed) are not well characterized. Characterizing this content over time could highlight over- and under-served problem domains, inspire new technological applications, and inform future research solicitation efforts. METHODS AND MATERIALS: This research sought to analyze and categorize historic AMTI proposals from 2010 to 2022 (n = 825). The analysis focused specifically on the "Problem to Be Solved" and "Technology to Demonstrated" sections of the proposals, whose categorizations are referred to as "Problem-Sets" and Solution-Sets" (PS and SS), respectively. A semi-supervised document clustering process was applied independently to the two sections. The process consisted of three stages: (1) Manual Document Annotation-a sample of proposals were manually labeled along each thematic axis; (2) Clustering-semi-supervised clustering, informed by the manually annotated sample, was applied to the proposals to produce document clusters; (3) Evaluation and Selection-quantitative and qualitative means were used to evaluate and select an optimal cluster solution. The results of the clustering were then summarized and presented descriptively. RESULTS: The results of the clustering process identified 24 unique PS and 20 unique SS. The most prevalent PS were Musculoskeletal Injury (12%), Traumatic Injury (11%), and Healthcare Systems Optimization (11%). The most prevalent SS were Sensing and Imaging Technology (27%), Virtual Health (23%), and Physical and Virtual Simulation (11.5%). The most common problem-solution pair was Healthcare Systems Optimization-Virtual Health, followed by Musculoskeletal Injury-Sensing and Imaging Technology. The analysis revealed that problem-solution-set co-occurrences were well distributed throughout the domain space, demonstrating the variety of research conducted in this research domain. CONCLUSIONS: A semi-supervised document clustering approach was applied to a repository of proposals to partially automate the process of document annotation. By applying this process, we successfully extracted thematic content from the proposals related to problems to be addressed and proposed technological solutions. This analysis provides a snapshot of the research supply in the domain of military medicine over the last 12 years. Future work should seek to replicate and improve the document clustering process used. Future efforts should also be made to compare these results to actual published work in the domain of military medicine, revealing differences in demand for research as determined by funding and publishing decision-makers and supply by researchers.

Research proposal content extraction using natural language processing and semi-supervised clustering: A demonstration and comparative analysis.

Funding institutions often solicit text-based research proposals to evaluate potential recipients. Leveraging the information contained in these documents could help institutions understand the supply of research within their domain. In this work, an end-to-end methodology for semi-supervised document clustering is introduced to partially automate classification of research proposals based on thematic areas of interest. The methodology consists of three stages: (1) manual annotation of a document sample; (2) semi-supervised clustering of documents; (3) evaluation of cluster results using quantitative metrics and qualitative ratings (coherence, relevance, distinctiveness) by experts. The methodology is described in detail to encourage replication and is demonstrated on a real-world data set. This demonstration sought to categorize proposals submitted to the US Army Telemedicine and Advanced Technology Research Center (TATRC) related to technological innovations in military medicine. A comparative analysis of method features was performed, including unsupervised vs. semi-supervised clustering, several document vectorization techniques, and several cluster result selection strategies. Outcomes suggest that pretrained Bidirectional Encoder Representations from Transformers (BERT) embeddings were better suited for the task than older text embedding techniques. When comparing expert ratings between algorithms, semi-supervised clustering produced coherence ratings ~ 25% better on average compared to standard unsupervised clustering with negligible differences in cluster distinctiveness. Last, it was shown that a cluster result selection strategy that balances internal and external validity produced ideal results. With further refinement, this methodological framework shows promise as a useful analytical tool for institutions to unlock hidden insights from untapped archives and similar administrative document repositories. Supplementary Information: The online version contains supplementary material available at 10.1007/s11192-023-04689-3.

Evaluating Medic Performance in Combat Casualty Care Simulation and Training: A Scoping Review of Prospective Research.

INTRODUCTION: Combat medics are required to perform highly technical medical procedures in austere environments with minimal error. Effective means to quantify medic performance in field and simulated environments are critical to optimize medic training procedures as well as to evaluate the influence of medical equipment and other supportive technologies on medic performance. Human performance evaluation in combat casualty care presents many unique challenges due to the unique environment (battlefields) and population (medics) that must be represented. Recent advances in simulation and measurement technology have presented opportunities to improve simulation fidelity and measurement quality; however, it is currently unclear to what extent these advances have been adopted in this domain. METHODOLOGY: In this work, a scoping review of recent (2011-2021) prospective research on Army medic (68 W and Special Operations) performance is presented. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines served as the framework for the review. The goal of this work was to summarize recent trends and practices and to illuminate opportunities for future work. Two human factors experts performed an exhaustive review of relevant, peer-reviewed literature and systematically identified articles for inclusion in the final analysis. The articles were examined in detail, and data elements of interest were extracted. RESULTS: Forty-eight articles were identified based on the defined inclusion criteria. Thirty three of the articles focused on technological evaluation, 25 focused on medic training procedures, and 5 focused on evaluating medical techniques. Study contributions were predominantly related to medic training materials/procedures and simulator technology. Supportive medical technologies, including telemedical systems, hemorrhage control devices, and ultrasound devices, also received significant attention. Timing was the most common metric used to quantify medic performance, followed by skill pass/fail ratings. There was a notable lack of neurophysiological data used to examine medic physical/cognitive workload during procedures, a growing practice in many other related domains. The most commonly simulated procedures were hemorrhage control, airway management, and thoracostomy. Notable limitations cited across articles were insufficient simulation fidelity, inadequate sample size or sample representativeness, and poor study design. CONCLUSIONS: This work provided a summary of recent peer-reviewed research related to medic simulation and training, and performance evaluation. This article should be used to contextualize existing research and inspire new research questions. Expanding and advancing research on medic simulation and training will help to ensure optimal casualty care at the front lines.

Utilizing Deep Learning for Detecting Adverse Drug Events in Structured and Unstructured Regulatory Drug Data Sets.

BACKGROUND: The US Food and Drug Administration (FDA) collects and retains several data sets on post-market drugs and associated adverse events (AEs). The FDA Adverse Event Reporting System (FAERS) contains millions of AE reports submitted by the public when a medication is suspected to have caused an AE. The FDA monitors these reports to identify drug safety issues that were undetected during the premarket evaluation of these products. These reports contain patient narratives that provide information regarding the AE that needs to be coded using standardized terminology to enable aggregation of reports for further review. Additionally, the FDA collects structured drug product labels (SPLs) that facilitate standardized distribution of information regarding marketed medical products. Manufacturers are currently not required to code labels with associated AEs. OBJECTIVES: Approaches for automated classification of reports by preferred terminology could enhance regulatory efficiency. The goal of this work was to assess the suitability of manually annotated FDA FAERS and SPL data sets to be subjected to predictive modeling. METHODS: A recurrent neural network (RNN) was proposed as a proof-of-concept model for automated extraction of preferred AE terminology. A separate RNN was fit and cross-validated on two regulatory data sets with varying properties. First, the researchers trained and cross-validated a model on 325 annotated FAERS patient narratives for a sample of AE terms. A model was then trained and validated on a data set of 100 SPLs. RESULTS: Model cross-validation results for product labels demonstrated that the model performed at least as well as more conventional models for all but one of the terms selected based on F1-score. Model results for the FAERS data set were mixed. CONCLUSIONS: This work successfully demonstrated a proof-of-concept machine learning approach to automatically detect AEs in several textual regulatory data sets to support post-market regulatory activities. Limited instances of each AE class likely prohibited models from generalizing data effectively. Additional data may permit more robust validation.

Cognitive task analysis and workload classification.

Automation can be utilized to relieve humans of difficult and repetitive tasks in many domains, presenting the opportunity for safer and more efficient systems. This increase in automation has led to new supervisory roles for human operators where humans monitor feedback from autonomous systems and provide input when necessary. Optimizing these roles requires tools for evaluation of task complexity and resulting operator cognitive workload. Cognitive task analysis is a process for modeling the cognitive actions required of a human during a task. This work presents an enhanced version of this process: Cognitive Task Analysis and Workload Classification (CTAWC). The goal of developing CTAWC was to provide a standardized process to decompose cognitive tasks in enough depth to allow for precise identification of sources of cognitive workload. CTAWC has the following advantages over conventional CTA methodology:•Integrates standard terminology from existing taxonomies for task classification to describe expected operator cognitive workload during task performance.•Provides a framework to evaluate adequate cognitive depth when decomposing cognitive tasks.•Provides a standard model upon which to build an empirical study to evaluate task complexity.