Towards multimodal graph neural networks for surgical instrument anticipation.

PURPOSE: Decision support systems and context-aware assistance in the operating room have emerged as the key clinical applications supporting surgeons in their daily work and are generally based on single modalities. The model- and knowledge-based integration of multimodal data as a basis for decision support systems that can dynamically adapt to the surgical workflow has not yet been established. Therefore, we propose a knowledge-enhanced method for fusing multimodal data for anticipation tasks. METHODS: We developed a holistic, multimodal graph-based approach combining imaging and non-imaging information in a knowledge graph representing the intraoperative scene of a surgery. Node and edge features of the knowledge graph are extracted from suitable data sources in the operating room using machine learning. A spatiotemporal graph neural network architecture subsequently allows for interpretation of relational and temporal patterns within the knowledge graph. We apply our approach to the downstream task of instrument anticipation while presenting a suitable modeling and evaluation strategy for this task. RESULTS: Our approach achieves an F1 score of 66.86% in terms of instrument anticipation, allowing for a seamless surgical workflow and adding a valuable impact for surgical decision support systems. A resting recall of 63.33% indicates the non-prematurity of the anticipations. CONCLUSION: This work shows how multimodal data can be combined with the topological properties of an operating room in a graph-based approach. Our multimodal graph architecture serves as a basis for context-sensitive decision support systems in laparoscopic surgery considering a comprehensive intraoperative operating scene.

Modification of Anaerobic Digestion Model No. 1 for modeling anaerobic digestion of cattle manure with changing solids retention time.

In this study, Anaerobic Digestion Model No.1 (ADM1) was modified to incorporate changes in biochemical parameters due to solids retention time (SRT) variations. Cattle manure (CM) and thermally hydrolyzed CM were selected for testing. Continuous anaerobic digestion reactors were operated under different SRT conditions ranging from 6.6 to 36.0 days for both samples. The biochemical parameters (kch, kli, kpr, um,ac, um,bu, um,pro, um,va, Kac, Kbu, Kpro, and Kva) for each SRT condition were determined. To modify ADM1, the equations obtained through linear regression were substituted into biochemical parameters as a function of SRT. The modified ADM1 demonstrated superior accuracy compared with conventional ADM1. This study implies the feasibility of optimizing biochemical parameters for modeling in response to changes in environmental variables.

A surgical activity model of laparoscopic cholecystectomy for co-operation with collaborative robots.

BACKGROUND: Laparoscopic cholecystectomy is a very frequent surgical procedure. However, in an ageing society, less surgical staff will need to perform surgery on patients. Collaborative surgical robots (cobots) could address surgical staff shortages and workload. To achieve context-awareness for surgeon-robot collaboration, the intraoperative action workflow recognition is a key challenge. METHODS: A surgical process model was developed for intraoperative surgical activities including actor, instrument, action and target in laparoscopic cholecystectomy (excluding camera guidance). These activities, as well as instrument presence and surgical phases were annotated in videos of laparoscopic cholecystectomy performed on human patients (n = 10) and on explanted porcine livers (n = 10). The machine learning algorithm Distilled-Swin was trained on our own annotated dataset and the CholecT45 dataset. The validation of the model was conducted using a fivefold cross-validation approach. RESULTS: In total, 22,351 activities were annotated with a cumulative duration of 24.9 h of video segments. The machine learning algorithm trained and validated on our own dataset scored a mean average precision (mAP) of 25.7% and a top K = 5 accuracy of 85.3%. With training and validation on our dataset and CholecT45, the algorithm scored a mAP of 37.9%. CONCLUSIONS: An activity model was developed and applied for the fine-granular annotation of laparoscopic cholecystectomies in two surgical settings. A machine recognition algorithm trained on our own annotated dataset and CholecT45 achieved a higher performance than training only on CholecT45 and can recognize frequently occurring activities well, but not infrequent activities. The analysis of an annotated dataset allowed for the quantification of the potential of collaborative surgical robots to address the workload of surgical staff. If collaborative surgical robots could grasp and hold tissue, up to 83.5% of the assistant's tissue interacting tasks (i.e. excluding camera guidance) could be performed by robots.

Evaluation of aircraft deicing fluid as an external carbon source for denitrification.

Water resource recovery facilities (WRRFs) performing biological nitrogen removal (BNR) often require external carbon sources for meeting nitrogen discharge permit limits. This brings an additional financial burden to the facilities considering the continuous need of these external carbon sources. This paper evaluates the utilization of airport stormwater, which in the winter season is rich in aircraft deicing fluid (ADF) as an alternative external carbon source. Denitrification and nitrification bench scale experiments were performed to assess the efficacy of external carbon sources to remove nitrogen in WRRFs. Experimental results showed that ADFs achieve denitrification rates of 0.064-0.066 d-1, higher than what achieved by a commercial carbon source, MicroC 2000A, with corresponding value of 0.058 d-1 at low temperatures, as low as 13 °C, which is considered a worst-case scenario for nitrogen removal efficiency. Furthermore, no inhibition to nitrification associated with the ADFs was observed. Subsequently a dynamic modeling study was conducted to assess the performance of ADFs as external carbon sources for denitrification and compared them to the conventional source that was being used in a full-scale BNR process. Results from the dynamic modeling study revealed that if 40 % of the spent-ADF at LaGuardia airport, New York City, could be collected with the stormwater and conveyed to a WRRF via the sewer collection system, an approximate reduction of 30 % of the commercial external carbon source could be accomplished by repurposing a waste product. This study contributes to the potential of ADF as a denitrification aid and an alternative for commercially available carbon sources with comparable nitrogen removal efficiencies.

Development and modeling of an integrated fixed-film activated sludge (IFAS) system for simultaneous nitrogen and carbon removal from an industrial estate wastewater.

The potential of an integrated fixed film activated sludge (IFAS) bioreactor for developing simultaneous aerobic and anoxic micro-zones under continuous aeration regime to promote carbon and nitrogen removal from Faraman industrial estate wastewater was evaluated in the present research. The effects of three independent variables on carbon and nitrogen removal were assessed. Overall, the optimum condition with 94 %, 77 %, and 2 NTU of COD (chemical oxygen demand) removal, Total nitrogen (TN) removal, and effluent turbidity has been specified with hydraulic retention time (HRT) of 11 h, air flow rate (AFR) of 3.5 L/min, and filling ratio (FR) of 50 %. To assess the stability of treating processes in the system, the IFAS system was operated in this optimal condition. Moreover, the simulation of the bioreactor was accomplished via calibration and verification of GPS-X model. GPSX simulation results and experimental data were compared using an independent sample T-test, which the T-test result confirmed that there was no significant difference between them.

The Association between Cesarean Section Delivery and Child Behavior: Is It Mediated by Maternal Post-Traumatic Stress Disorder and Maternal Postpartum Depression?

Cesarean sections (C-sections) account for up to 21% of births worldwide. Studies have linked delivery via C-section with an increased risk of child behavior problems, such as internalizing and externalizing behaviors. Maternal postpartum depression (PPD) is also linked to child behavioral problems and may play a mediating role in the association between the mode of delivery and child behavior. Mixed findings between mode of delivery and PPD may be due to a failure to distinguish between C-section types, as unplanned/emergency C-sections are linked to post-traumatic stress disorder (PTSD), which has been linked to PPD. The objectives of this study were to determine whether, (1) compared with spontaneous vaginal delivery (SVD) and planned C-section, unplanned/emergency C-sections are associated with increased child behavior problems at two to three years of age and (2) maternal PTSD and PPD mediate the association between delivery type and child behavior problems. A secondary data analysis was conducted on 938 mother-child dyads enrolled in the Alberta Pregnancy Outcomes and Nutrition (APrON) study. Conditional process modeling was employed. Child behavior was assessed using the Child Behavior Checklist (CBCL) 1.5-5 years, and maternal PPD and PTSD were assessed using the Edinburgh Postnatal Depression Scale (EPDS) and the Psychiatric Diagnostic Screening Questionnaire (PDSQ), respectively. No associations were found between delivery type and child behaviors; however, the indirect effect of emergency C-section on child behaviors was significant via the mediating pathway of maternal PTSD on PPD symptoms.

Toward an interoperable, intraoperative situation recognition system via process modeling, execution, and control using the standards BPMN and CMMN.

PURPOSE: For the modeling, execution, and control of complex, non-standardized intraoperative processes, a modeling language is needed that reflects the variability of interventions. As the established Business Process Model and Notation (BPMN) reaches its limits in terms of flexibility, the Case Management Model and Notation (CMMN) was considered as it addresses weakly structured processes. METHODS: To analyze the suitability of the modeling languages, BPMN and CMMN models of a Robot-Assisted Minimally Invasive Esophagectomy and Cochlea Implantation were derived and integrated into a situation recognition workflow. Test cases were used to contrast the differences and compare the advantages and disadvantages of the models concerning modeling, execution, and control. Furthermore, the impact on transferability was investigated. RESULTS: Compared to BPMN, CMMN allows flexibility for modeling intraoperative processes while remaining understandable. Although more effort and process knowledge are needed for execution and control within a situation recognition system, CMMN enables better transferability of the models and therefore the system. Concluding, CMMN should be chosen as a supplement to BPMN for flexible process parts that can only be covered insufficiently by BPMN, or otherwise as a replacement for the entire process. CONCLUSION: CMMN offers the flexibility for variable, weakly structured process parts, and is thus suitable for surgical interventions. A combination of both notations could allow optimal use of their advantages and support the transferability of the situation recognition system.

Modeling the γ-irradiation inactivation kinetics of foodborne pathogens Escherichia coli O157:H7, Salmonella, Staphylococcus aureus and Bacillus cereus in instant soup.

The objective of the present study was to assess the inactivation kinetics of γ-irradiation of selected foodborne pathogens in instant soup. Escherichia coli O157:H7 (ATCC 25922), Salmonella enterica subsp. enterica serovar Enteritidis (ATCC 13076), Staphylococcus aureus (ATCC 2592), and Bacillus cereus (ATCC 11778) were inoculated into instant soup and irradiated at various doses of 0 (control), 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 4.0, 5.0, and 10.0 kGy using 60Co source. The radiation response of these four major foodborne disease pathogens in instant soup was tested. As expected, the pathogen population decreased with increasing irradiation dose. By comparing bacterial resistance in instant soups according to D10 values, E coli O157: H7 was the most radio-resistant bacteria (D10 of 1.580 kGy), followed by Salmonella (D10 of 1.160 kGy), S aureus (D10 of 0.775 kGy), B cereus (D10 of 0.462 kGy). For modeling of inactivation kinetics, both, the conventional first-order linear model and Weibull model were compared and the goodness of fit of these models was investigated. Weibull model produced a better fit to the data. This research has shown that γ-irradiation was effective to eliminate pathogens in instant soup and it can be a great way to assure the microbiological safety of the instant soup.

Physics-informed neural networks to solve lumped kinetic model for chromatography process.

Numerical method is widely used for solving the mechanistic models of chromatography process, but it is time-consuming and hard to response in real-time. Physics-informed neural network (PINN) as an emerging technology combines the structure of neural network with physics laws, and is getting noticed for solving physics problems with a balanced accuracy and calculation speed. In this research, a proof-of-concept study was carried out to apply PINN to chromatography process simulation. The PINN model structure was designed for the lumped kinetic model (LKM) with all LKM parameters. The PINN structure, training data and model complexity were optimized, and an optimal mode was obtained by adopting an in-series structure with a nonuniform training data set focusing on the breakthrough transition region. A PINN for LKM (LKM-PINN) consisting of four neural networks, 12 layers and 606 neurons was then used for the simulation of breakthrough curves of chromatography processes. The LKM parameters were estimated with two breakthrough curves and used to infer the breakthrough curves at different residence times, loading concentrations and column sizes. The results were comparable to that obtained with numerical methods. With the same raw data and constraints, the average fitting error for LKM-PINN model was 0.075, which was 0.081 for numerical method. With the same initial guess, the LKM-PINN model took 160 s to complete the fitting, while the numerical method took 7 to 72 min, depending on the fitting settings. The fitting speed of LKM-PINN model was further improved to 30 s with random initial guess. Thus, the LKM-PINN model developed in this study is capable to be applied to real-time simulation for digital twin.

Online Monitoring of Sourdough Fermentation Using a Gas Sensor Array with Multivariate Data Analysis.

Sourdough can improve bakery products' shelf life, sensory properties, and nutrient composition. To ensure high-quality sourdough, the fermentation has to be monitored. The characteristic process variables for sourdough fermentation are pH and the degree of acidity measured as total titratable acidity (TTA). The time- and cost-intensive offline measurement of process variables can be improved by utilizing online gas measurements in prediction models. Therefore, a gas sensor array (GSA) system was used to monitor the fermentation process of sourdough online by correlation of exhaust gas data with offline measurement values of the process variables. Three methods were tested to utilize the extracted features from GSA to create the models. The most robust prediction models were achieved using a PCA (Principal Component Analysis) on all features and combined two fermentations. The calibrations with the extracted features had a percentage root mean square error (RMSE) from 1.4% to 12% for the pH and from 2.7% to 9.3% for the TTA. The coefficient of determination (R2) for these calibrations was 0.94 to 0.998 for the pH and 0.947 to 0.994 for the TTA. The obtained results indicate that the online measurement of exhaust gas from sourdough fermentations with gas sensor arrays can be a cheap and efficient application to predict pH and TTA.

Multipart Build Effects on Temperature and Residual Stress by Laser Beam Powder Bed Fusion Additive Manufacturing.

Laser beam powder bed fusion (PBF-LB) is a leading technique among metal additive manufacturing (AM), and it has a wide range of applications in aerospace and medical devices. Most of the existing PBF-LB process modeling is mainly based on the fabrication of a single part on a large build plate, which is not reflective of the practical multipart PBF-LB manufacturing. The effects of batch size on the thermal and mechanical behavior of additively manufactured parts have not been investigated. In this work, the multipart PBF-LB thermomechanical modeling framework was proposed for the first time. The effects of sample numbers (1, 2, and 4) on temperature and residual stress (RS) of part-scale components were computationally investigated. It is found that RS within the parts decreased with increasing number of components per build. Parts located at the central areas of the build plate had larger RS than at the border. These findings can be beneficial for informing AM designers and operators of the optimum printing setup to minimize RS of metal parts in PBF-LB.

Ki-Cook: clustering multimodal cooking representations through knowledge-infused learning.

Cross-modal recipe retrieval has gained prominence due to its ability to retrieve a text representation given an image representation and vice versa. Clustering these recipe representations based on similarity is essential to retrieve relevant information about unknown food images. Existing studies cluster similar recipe representations in the latent space based on class names. Due to inter-class similarity and intraclass variation, associating a recipe with a class name does not provide sufficient knowledge about recipes to determine similarity. However, recipe title, ingredients, and cooking actions provide detailed knowledge about recipes and are a better determinant of similar recipes. In this study, we utilized this additional knowledge of recipes, such as ingredients and recipe title, to identify similar recipes, emphasizing attention especially on rare ingredients. To incorporate this knowledge, we propose a knowledge-infused multimodal cooking representation learning network, Ki-Cook, built on the procedural attribute of the cooking process. To the best of our knowledge, this is the first study to adopt a comprehensive recipe similarity determinant to identify and cluster similar recipe representations. The proposed network also incorporates ingredient images to learn multimodal cooking representation. Since the motivation for clustering similar recipes is to retrieve relevant information for an unknown food image, we evaluated the ingredient retrieval task. We performed an empirical analysis to establish that our proposed model improves the Coverage of Ground Truth by 12% and the Intersection Over Union by 10% compared to the baseline models. On average, the representations learned by our model contain an additional 15.33% of rare ingredients compared to the baseline models. Owing to this difference, our qualitative evaluation shows a 39% improvement in clustering similar recipes in the latent space compared to the baseline models, with an inter-annotator agreement of the Fleiss kappa score of 0.35.

Development of ANN-Based Warpage Prediction Model for FCCSP via Subdomain Sampling and Taguchi Hyperparameter Optimization.

This study aims to establish an accurate prediction model using artificial neural networks (ANNs) to effectively and efficiently predict the process-induced warpage of a flip-chip chip-scale package (FCCSP). To enhance model performance, a novel subdomain-based sampling strategy and Taguchi hyperparameter optimization are proposed in the ANN algorithm. To simulate the warpage behavior the FCCSP during fabrication, a process modeling approach is proposed, where the viscoelastic behavior of the epoxy molding compound is included, in which the viscoelastic properties are determined using dynamic mechanical measurement. In addition, the temperature-dependent thermal-mechanical properties of the materials in the FCCSP are assessed through thermal-mechanical analysis and dynamic mechanical analysis. The modeled warpage results are verified by the warpage measurement. Next, warpage parametric analysis is performed to identify the key factors most affecting warpage behavior for use in the construction of the warpage prediction model. Moreover, the advantages of the proposed sampling and hyperparameter tuning approaches are proved by comparing with other existing models, and the validity of the developed ANN-based deep learning warpage prediction model is demonstrated through a validation dataset.

Ensemble Methods for APS In-Flight Particle Temperature and Velocity Prediction Considering Torch Electrodes Ageing.

The nonlinear relationship between the input process parameters and in-flight particle characteristics of the atmospheric plasma spray (APS) is of paramount importance for coating properties design and quality. It is also known that the ageing of torch electrodes affects this relationship. In recent years, machine learning algorithms have proven to be able to take into account such complex nonlinear interactions. This work illustrates the application of ensemble methods to predict the in-flight particle temperature and velocity during an APS process considering torch electrodes ageing. Experiments were performed to record simultaneously the input process parameters, the in-flight powder particle characteristics and the electrodes usage time. Random Forest (RF) and Gradient Boosting (GB) were used to rank and select the features for the APS process data recorded as the electrodes aged and the corresponding predictive models were compared. The time series aspect of the multivariate APS in-flight particle characteristics data is explored. Two strategies of time series embedding are considered. The first one simply embeds the attributes and the targets from the previous n time segments considered without any modification; whereas the second strategy first performs differencing to make the time series stationary before embedding. For the present application, RF is found to be more suitable than GB since RF can predict both the in-flight particle velocity and temperature simultaneously, properly considering the interactions between the two targets. On the other hand, GB can only predict these two targets one at a time. The superior performance of both embedded predictive models and the feature rankings of them suggest that it is better to consider the APS data as time series for the in-flight particle characteristic prediction. In particular, it is demonstrated that it is advantageous to first make the time series stationary using the traditional differencing technique, even when modeling using RF.

[Research progress and maturity assessment of continuous manufacturing of traditional Chinese medicine].

The pharmaceutical manufacturing model is gradually changing from intermittent manufacturing to continuous manufacturing and intelligent manufacturing. This paper briefly reviewed the supervision and research progress in continuous pharmaceutical manufacturing in China and abroad and described the definition and advantages of continuous pharmaceutical manufacturing. The continuous manufacturing of traditional Chinese medicine(TCM) at the current stage was summarized in the following three terms: the enhancement of the continuity of intermittent manufacturing operations, the integration of continuous equipment to improve physical continuity between units, and the application of advanced process control strategies to improve process continuity. To achieve continuous manufacturing of TCM, the corresponding key technologies, such as material property characterization, process modeling and simulation, process analysis technology, and system integration, were analyzed from the process and equipment, respectively. It was proposed that the continuous manufacturing equipment system should have the characteristics of high speed, high response, and high reliability, "three high(H~3)" for short. Considering the characteristics and current situation of TCM manufacturing, based on the two dimensions of product quality control and production efficiency, a maturity assessment model for continuous manufacturing of TCM, consisting of operation continuity, equipment continuity, process continuity, and quality control continuity, was proposed to provide references for the application of continuous manufacturing technology for TCM. The implementation of continuous manufacturing or the application of key continuous manufacturing technologies in TCM can help to systematically integrate advanced pharmaceutical technology elements and promote the uniformity of TCM quality and the improvement of production efficiency.

Process model analysis of parenchyma sparing laparoscopic liver surgery to recognize surgical steps and predict impact of new technologies.

BACKGROUND: Surgical process model (SPM) analysis is a great means to predict the surgical steps in a procedure as well as to predict the potential impact of new technologies. Especially in complicated and high-volume treatments, such as parenchyma sparing laparoscopic liver resection (LLR), profound process knowledge is essential for enabling improving surgical quality and efficiency. METHODS: Videos of thirteen parenchyma sparing LLR were analyzed to extract the duration and sequence of surgical steps according to the process model. The videos were categorized into three groups, based on the tumor locations. Next, a detailed discrete events simulation model (DESM) of LLR was built, based on the process model and the process data obtained from the endoscopic videos. Furthermore, the impact of using a navigation platform on the total duration of the LLR was studied with the simulation model by assessing three different scenarios: (i) no navigation platform, (ii) conservative positive effect, and (iii) optimistic positive effect. RESULTS: The possible variations of sequences of surgical steps in performing parenchyma sparing depending on the tumor locations were established. The statistically most probable chain of surgical steps was predicted, which could be used to improve parenchyma sparing surgeries. In all three categories (i-iii) the treatment phase covered the major part (~ 40%) of the total procedure duration (bottleneck). The simulation results predict that a navigation platform could decrease the total surgery duration by up to 30%. CONCLUSION: This study showed a DESM based on the analysis of steps during surgical procedures can be used to predict the impact of new technology. SPMs can be used to detect, e.g., the most probable workflow paths which enables predicting next surgical steps, improving surgical training systems, and analyzing surgical performance. Moreover, it provides insight into the points for improvement and bottlenecks in the surgical process.

Diagnosing and characterizing the mechanisms of biological phosphorus removal at the Great Lakes Water Authority (GLWA) water resource recovery facility (WRRF).

Previous research has demonstrated that biological phosphorus removal (bio-P) occurs in the Great Lakes Water Authority (GLWA) water resource recovery facility (WRRF) high purity oxygen activated sludge (HPO-AS) process, suggesting that sludge fermentation in the secondary clarifier sludge blanket is key to bio-P occurrence. This study, combining batch reactor testing, the development of a process model for the HPO-AS process using Sumo21 (Dynamita), and the analysis of eight and a half years of plant operating data, showed that bio-P consistently occurs at the GLWA WRRF. This occurrence is attributed to the unique configuration of the HPO-AS process, which has a relatively large secondary clarifier compared to the bioreactor, and the characteristics of the influent wastewater, primarily particulate matter with limited concentrations of dissolved biodegradable organic matter. The volatile fatty acids (VFAs) needed for polyphosphate accumulating organisms (PAOs) growth are produced in the secondary clarifier sludge blanket, which provides more than four times the anaerobic biomass inventory compared to the anaerobic zones in the bioreactor, thus facilitating bio-P in the current system. Opportunities exist to further optimize the phosphorus removal performance of the HPO-AS process and reduce the amount of ferric chloride used. These findings may be of interest to researchers investigating biological phosphorus removal in similar systems. PRACTITIONER POINTS: Fermentation in the clarifier sludge blanket an essential component of bio-P process at this facility. Results suggest simple adjustments to the system could lead to further improvements in bio-P. It is possible to decrease the use of chemical phosphorus removal methods (i.e., ferric chloride) while simultaneously increasing bio-P. Determining the phosphorus mass balance from sludge streams provides insight into evaluating the effectiveness of the phosphorus recovery system.

Control oriented modeling of twin-screw granulation in the ConsiGmaTM-25 production plant.

ConsiGmaTM-25 is a continuous production plant integrating a twin-screw granulation, fluid bed drying, granule conditioning, and a tableting unit. The particle size distribution (PSD), active pharmaceutical ingredient (API) content, and liquid content of wet granules after twin-screw granulation affect the quality of intermediate and final products. This paper proposes methods for real-time monitoring of these quantities and control-oriented modeling of the granulator. The PSD of wet granules is monitored via an in-line process analytical technology (PAT) probe based on the spatial velocimetry principle. The algorithm for signal processing and evaluation of PSD characteristics is developed and applied to the acquired PSD data. A dynamic process model predicting PSD characteristics from granulation parameters is trained via the local linear model tree (LoLiMoT) approach. The experimental data required for the model training are collected via systematically designed excitation runs. Finally, the performance of the identified model is examined and verified by means of a new set of validation runs. Furthermore, an in-line PAT probe based on Raman spectroscopy is developed and integrated after the granulator. The API- and liquid content of produced wet granules are evaluated from the spectral data by means of chemometric modeling, and chemometric models are validated on a separate set of experimental data. The solutions proposed in this research can be used as a reliable (and necessary) basis for the development of advanced quality-by-design control concepts (e.g., PSD process control). Such concepts would ultimately improve the ConsiGmaTM-25 process performance in terms of robustness against disturbances and quality of intermediate and final products.

COMPASS: a formal framework and aggregate dataset for generalized surgical procedure modeling.

PURPOSE: We propose a formal framework for the modeling and segmentation of minimally invasive surgical tasks using a unified set of motion primitives (MPs) to enable more objective labeling and the aggregation of different datasets. METHODS: We model dry-lab surgical tasks as finite state machines, representing how the execution of MPs as the basic surgical actions results in the change of surgical context, which characterizes the physical interactions among tools and objects in the surgical environment. We develop methods for labeling surgical context based on video data and for automatic translation of context to MP labels. We then use our framework to create the COntext and Motion Primitive Aggregate Surgical Set (COMPASS), including six dry-lab surgical tasks from three publicly available datasets (JIGSAWS, DESK, and ROSMA), with kinematic and video data and context and MP labels. RESULTS: Our context labeling method achieves near-perfect agreement between consensus labels from crowd-sourcing and expert surgeons. Segmentation of tasks to MPs results in the creation of the COMPASS dataset that nearly triples the amount of data for modeling and analysis and enables the generation of separate transcripts for the left and right tools. CONCLUSION: The proposed framework results in high quality labeling of surgical data based on context and fine-grained MPs. Modeling surgical tasks with MPs enables the aggregation of different datasets and the separate analysis of left and right hands for bimanual coordination assessment. Our formal framework and aggregate dataset can support the development of explainable and multi-granularity models for improved surgical process analysis, skill assessment, error detection, and autonomy.

A Modeling Framework to Examine Psychological Processes Underlying Ordinal Responses and Response Times of Psychometric Data.

This article presents a joint modeling framework of ordinal responses and response times (RTs) for the measurement of latent traits. We integrate cognitive theories of decision-making and confidence judgments with psychometric theories to model individual-level measurement processes. The model development starts with the sequential sampling framework which assumes that when an item is presented, a respondent accumulates noisy evidence over time to respond to the item. Several cognitive and psychometric theories are reviewed and integrated, leading us to three psychometric process models with different representations of the cognitive processes underlying the measurement. We provide simulation studies that examine parameter recovery and show the relationships between latent variables and data distributions. We further test the proposed models with empirical data measuring three traits related to motivation. The results show that all three models provide reasonably good descriptions of observed response proportions and RT distributions. Also, different traits favor different process models, which implies that psychological measurement processes may have heterogeneous structures across traits. Our process of model building and examination illustrates how cognitive theories can be incorporated into psychometric model development to shed light on the measurement process, which has had little attention in traditional psychometric models.