Automated algorithm for medical data structuring, and segmentation using artificial intelligence within secured environment for dataset creation.

Objective: Routinely collected electronic health records using artificial intelligence (AI)-based systems bring out enormous benefits for patients, healthcare centers, and its industries. Artificial intelligence models can be used to structure a wide variety of unstructured data. Methods: We present a semi-automatic workflow for medical dataset management, including data structuring, research extraction, AI-ground truth creation, and updates. The algorithm creates directories based on keywords in new file names. Results: Our work focuses on organizing computed tomography (CT), magnetic resonance (MR) images, patient clinical data, and segmented annotations. In addition, an AI model is used to generate different initial labels that can be edited manually to create ground truth labels. The manually verified ground truth labels are later included in the structured dataset using an automated algorithm for future research. Conclusion: This is a workflow with an AI model trained on local hospital medical data with output based/adapted to the users and their preferences. The automated algorithms and AI model could be implemented inside a secondary secure environment in the hospital to produce inferences.

Hybrid optical-vision tracking in laparoscopy: accuracy of navigation and ultrasound reconstruction.

INTRODUCTION: The use of laparoscopic and robotic liver surgery is increasing. However, it presents challenges such as limited field of view and organ deformations. Surgeons rely on laparoscopic ultrasound (LUS) for guidance, but mentally correlating ultrasound images with pre-operative volumes can be difficult. In this direction, surgical navigation systems are being developed to assist with intra-operative understanding. One approach is performing intra-operative ultrasound 3D reconstructions. The accuracy of these reconstructions depends on tracking the LUS probe. MATERIAL AND METHODS: This study evaluates the accuracy of LUS probe tracking and ultrasound 3D reconstruction using a hybrid tracking approach. The LUS probe is tracked from laparoscope images, while an optical tracker tracks the laparoscope. The accuracy of hybrid tracking is compared to full optical tracking using a dual-modality tool. Ultrasound 3D reconstruction accuracy is assessed on an abdominal phantom with CT transformed into the optical tracker's coordinate system. RESULTS: Hybrid tracking achieves a tracking error < 2 mm within 10 cm between the laparoscope and the LUS probe. The ultrasound reconstruction accuracy is approximately 2 mm. CONCLUSION: Hybrid tracking shows promising results that can meet the required navigation accuracy for laparoscopic liver surgery.

Ascending with ultrasound: telementored eFAST in flight-a feasibility study.

PURPOSE: Teleultrasound uses telecommunication technologies to transmit ultrasound images from a remote location to an expert who guides the acquisition of images and interprets them in real time. Multiple studies have demonstrated the feasibility of teleultrasound. However, its application during helicopter flight using long-term evolution (LTE) for streaming has not been studied. Therefore, we conducted a study to examine the feasibility of teleultrasound in an Airbus H145 helicopter. METHODS: Four anesthesiologists and one military physician were recruited to perform telementored extended Focused Assessment with Sonography in Trauma (eFAST) during nine helicopter flights, each with a unique healthy volunteer. A radiologist was recruited as a remote expert, guiding the physicians in their examinations. The examining physicians reported the user experience of telementored eFAST on a questionnaire, while the remote expert rated the diagnostic quality of the images on a 1-5 Likert scale. In addition, we measured the duration of the examinations and key LTE network parameters including signal strength, quality, and continuity. RESULTS: The images were rated to an average of 4.9 by the remote expert, corresponding to good diagnostic quality. The average duration of telementored eFAST was 05:54 min. LTE coverage was negatively affected by proximity to urban areas and ceased above 2000 ft altitude. Occasional audio problems were addressed by using the Voice over LTE network for communication. The examining physicians unanimously reported on the questionnaire that they would use telementored eFAST on patients. CONCLUSION: Telementored eFAST is feasible in ambulance helicopters and can produce images of good diagnostic quality. However, it relies on stable LTE coverage, which is influenced by many factors, including the helicopter's altitude and flight path. Furthermore, its benefit on patient outcomes remains to be proven.

Automated structuring of gait data for analysis purposes - A deep learning pilot example.

Clinical gait analysis can help diagnose ambulatory children with cerebral palsy and provide treatment recommendations. This group represents the largest group of children with gait problems. Currently, the workflow for 3D gait analysis involves a complex process of collecting motion capture data and other types of data, analyzing the collected data, and creating an expert knowledge-based assessment. With this in mind, a data pipeline is essential for efficiently and effectively structuring data and reducing the time and effort required for data annotation and organization.A novel data pipeline has been developed to help structure, anonymize and automate parts of the annotation process of the data. In this sense, a pilot experiment was conducted using a simple convolutional neural network to classify between hemi-plegic and diplegic gait. This experiment included preprocessing the data, training the model and testing it.The data pipeline was used to create a semi-automated annotated data set. The neural network was trained on the data set and achieved an accuracy of 0.78 and a median of 1.0 on a holdout test set.

Process model analysis of parenchyma sparing laparoscopic liver surgery to recognize surgical steps and predict impact of new technologies.

BACKGROUND: Surgical process model (SPM) analysis is a great means to predict the surgical steps in a procedure as well as to predict the potential impact of new technologies. Especially in complicated and high-volume treatments, such as parenchyma sparing laparoscopic liver resection (LLR), profound process knowledge is essential for enabling improving surgical quality and efficiency. METHODS: Videos of thirteen parenchyma sparing LLR were analyzed to extract the duration and sequence of surgical steps according to the process model. The videos were categorized into three groups, based on the tumor locations. Next, a detailed discrete events simulation model (DESM) of LLR was built, based on the process model and the process data obtained from the endoscopic videos. Furthermore, the impact of using a navigation platform on the total duration of the LLR was studied with the simulation model by assessing three different scenarios: (i) no navigation platform, (ii) conservative positive effect, and (iii) optimistic positive effect. RESULTS: The possible variations of sequences of surgical steps in performing parenchyma sparing depending on the tumor locations were established. The statistically most probable chain of surgical steps was predicted, which could be used to improve parenchyma sparing surgeries. In all three categories (i-iii) the treatment phase covered the major part (~ 40%) of the total procedure duration (bottleneck). The simulation results predict that a navigation platform could decrease the total surgery duration by up to 30%. CONCLUSION: This study showed a DESM based on the analysis of steps during surgical procedures can be used to predict the impact of new technology. SPMs can be used to detect, e.g., the most probable workflow paths which enables predicting next surgical steps, improving surgical training systems, and analyzing surgical performance. Moreover, it provides insight into the points for improvement and bottlenecks in the surgical process.

Real-time Remote Expert-guided Echocardiography by Medical Students.

BACKGROUND: Echocardiography is a highly specialised examination performed by experienced healthcare professionals. These experienced healthcare professionals may not be available to patients during all hours in rural healthcare facilities. Remote-guided echocardiography could improve the availability of specialised care for patients living in rural areas. This study examined the feasibility of real-time remote guidance for medical students to perform an echocardiographic assessment of the left side of the heart. Thirteen healthy volunteers were recruited for remote-guided echocardiography, which was performed by 13 medical students. Student examinations/images were compared to reference echocardiography. Measurements of left ventricular fractional shortening and mitral valve blood flow velocity were also compared. Furthermore, guidance through a smartphone videoconference was compared to designated remote guidance software. RESULTS: Two-thirds of the images acquired by students were rated as medium or good quality and usable to evaluate two thirds of the cardiac structures. No significant bias was found for left ventricular fractional shortening. The measurements from the students' exams had a variation coefficient of 14.8% compared to the reference. The calculated deviation of the insonation angle was above 25° for both E and A-wave mitral valve blood flow velocity measurements. Images acquired by guidance through smartphone videoconference were of lower quality than those obtained using the designated remote guidance software. CONCLUSION: Real-time remote-guided echocardiography performed by medical students has limited value for clinical screening but could be useful for educational purposes.

Detection of inflow obstruction in left ventricular assist devices by accelerometer: A porcine model study.

BACKGROUND: Left ventricular assist devices (LVAD) provide circulatory blood pump support for severe heart failure patients. Pump inflow obstructions may lead to stroke and pump malfunction. We aimed to verify in vivo that gradual inflow obstructions, representing prepump thrombosis, are detectable by a pump-attached accelerometer, where the routine use of pump power (PLVAD) is deficient. METHOD: In a porcine model (n = 8), balloon-tipped catheters obstructed HVAD inflow conduits by 34% to 94% in 5 levels. Afterload increases and speed alterations were conducted as controls. We computed nonharmonic amplitudes (NHA) of pump vibrations captured by the accelerometer for the analysis. Changes in NHA and PLVAD were tested by a pairwise nonparametric statistical test. Detection sensitivities and specificities were investigated by receiver operating characteristics with areas under the curves (AUC). RESULTS: NHA remained marginally affected during control interventions, unlike PLVAD. NHA elevated during obstructions within 52-83%, while mass pendulation was most pronounced. Meanwhile, PLVAD changed far less. Increased pump speeds tended to amplify the NHA elevations. The corresponding AUC was 0.85-1.00 for NHA and 0.35-0.73 for PLVAD. CONCLUSION: Elevated NHA provides a reliable indication of subclinical gradual inflow obstructions. The accelerometer can potentially supplement PLVAD for earlier warnings and localization of pump.

Pulse arrival time variation as a non-invasive marker of acute response to cardiac resynchronization therapy.

AIMS: Successful cardiac resynchronization therapy (CRT) shortens the pre-ejection period (PEP) which is prolonged in the left bundle branch block (LBBB). In a combined animal and patient study, we investigated if changes in the pulse arrival time (PAT) could be used to measure acute changes in PEP during CRT implantation and hence be used to evaluate acute CRT response non-invasively and in real time. METHODS AND RESULTS: In six canines, a pulse transducer was attached to a lower limb and PAT was measured together with left ventricular (LV) pressure by micromanometer at baseline, after induction of LBBB and during biventricular pacing. Time-to-peak LV dP/dt (Td) was used as a surrogate for PEP. In twelve LBBB patients during implantation of CRT, LV and femoral pressures were measured at baseline and during five different pacing configurations. PAT increased from baseline (277 ± 9 ms) to LBBB (313 ± 16 ms, P < 0.05) and shortened with biventricular pacing (290 ± 16 ms, P < 0.05) in animals. There was a strong relationship between changes in PAT and Td in patients (r2 = 0.91). Two patients were classified as non-responders at 6 months follow-up. CRT decreased PAT from 320 ± 41 to 298 ± 39 ms (P < 0.05) in the responders, while PAT increased by 5 and 8 ms in the two non-responders. CONCLUSION: This proof-of-concept study indicates that PAT can be used as a simple, non-invasive method to assess the acute effects of CRT in real time with the potential to identify long-term response in patients.

Learning deep abdominal CT registration through adaptive loss weighting and synthetic data generation.

PURPOSE: This study aims to explore training strategies to improve convolutional neural network-based image-to-image deformable registration for abdominal imaging. METHODS: Different training strategies, loss functions, and transfer learning schemes were considered. Furthermore, an augmentation layer which generates artificial training image pairs on-the-fly was proposed, in addition to a loss layer that enables dynamic loss weighting. RESULTS: Guiding registration using segmentations in the training step proved beneficial for deep-learning-based image registration. Finetuning the pretrained model from the brain MRI dataset to the abdominal CT dataset further improved performance on the latter application, removing the need for a large dataset to yield satisfactory performance. Dynamic loss weighting also marginally improved performance, all without impacting inference runtime. CONCLUSION: Using simple concepts, we improved the performance of a commonly used deep image registration architecture, VoxelMorph. In future work, our framework, DDMR, should be validated on different datasets to further assess its value.

Detection of inflow obstruction in left ventricular assist devices by accelerometer: An in vitro study.

Inflow obstruction in left ventricular assist devices (LVAD) may lead to embolic stroke and pump malfunction. We investigated if an accelerometer detected graded LVAD inflow obstructions. Detection performances were compared to the current continuous surveillance routine based on the pump power consumption (PLVAD). In ten mock circuit experiments, four different-sized pendulating balloons obstructed HVAD™ inflow conduits cross-section areas by 14%-75%. Nonharmonic amplitudes (NHA) of continuous signals from a triaxial accelerometer attached to the LVAD were compared against single-point PLVAD values, using load and speed alterations as control interventions. We analyzed the NHA band power with a pairwise nonparametric statistical test. The detection performances were analyzed by receiver operating characteristics with areas under the curves (AUC). The NHA remained unaffected during load alterations. In contrast, NHA increased significantly from the 27% obstruction level (AUC≥0.82), an effect amplified by increased pump speed. PLVAD did not change significantly below the maximal 75% obstruction level (AUC≤0.36). In conclusion, NHA detected the inflow obstructions much better than PLVAD. The technique may provide a future monitoring modality of any pendulating obstructive inflow pathology.

Evaluation of clinical applicability of automated liver parenchyma segmentation of multi-center magnetic resonance images.

Purpose: Automated algorithms for liver parenchyma segmentation can be used to create patient-specific models (PSM) that assist clinicians in surgery planning. In this work, we analyze the clinical applicability of automated deep learning methods together with level set post-processing for liver segmentation in contrast-enhanced T1-weighted magnetic resonance images. Methods: UNet variants with/without attention gate, multiple loss functions, and level set post-processing were used in the workflow. A multi-center, multi-vendor dataset from Oslo laparoscopic versus open liver resection for colorectal liver metastasis clinical trial is used in our study. The dataset of 150 volumes is divided as 81:25:25:19 corresponding to train:validation:test:clinical evaluation respectively. We evaluate the clinical use, time to edit automated segmentation, tumor regions, boundary leakage, and over-and-under segmentations of predictions. Results: The deep learning algorithm shows a mean Dice score of 0.9696 in liver segmentation, and we also examined the potential of post-processing to improve the PSMs. The time to create clinical use segmentations of level set post-processed predictions shows a median time of 16 min which is 2 min less than deep learning inferences. The intra-observer variations between manually corrected deep learning and level set post-processed segmentations show a 3% variation in the Dice score. The clinical evaluation shows that 7 out of 19 cases of both deep learning and level set post-processed segmentations contain all required anatomy and pathology, and hence these results could be used without any manual corrections. Conclusions: The level set post-processing reduces the time to create clinical standard segmentations, and over-and-under segmentations to a certain extent. The time advantage greatly supports clinicians to spend their valuable time with patients.

Generic surgical process model for minimally invasive liver treatment methods.

Surgical process modelling is an innovative approach that aims to simplify the challenges involved in improving surgeries through quantitative analysis of a well-established model of surgical activities. In this paper, surgical process model strategies are applied for the analysis of different Minimally Invasive Liver Treatments (MILTs), including ablation and surgical resection of the liver lesions. Moreover, a generic surgical process model for these differences in MILTs is introduced. The generic surgical process model was established at three different granularity levels. The generic process model, encompassing thirteen phases, was verified against videos of MILT procedures and interviews with surgeons. The established model covers all the surgical and interventional activities and the connections between them and provides a foundation for extensive quantitative analysis and simulations of MILT procedures for improving computer-assisted surgery systems, surgeon training and evaluation, surgeon guidance and planning systems and evaluation of new technologies.

Evaluation and Comparison of Target Registration Error in Active and Passive Optical Tracking Systems.

Optical tracking systems combined with imaging modalities such as computed tomography and magnetic reso-nance imaging are important parts of image guided surgery systems. By determining the location and orientation of sur-gical tools relative to a patient's reference system, tracking systems assist surgeons during the planning and execution of image guided procedures. Therefore, knowledge of the tracking system-induced error is of great importance. To this end, this study compared one passive and two active optical tracking systems in terms of their Target Registration Error. Two experiments were performed to measure the systems' accuracy, testing the impact of factors such as the size of the measuring volume, length of surgical instruments and environmental conditions with orthopedic procedures in mind. According to the performed experiments, the active systems achieved significantly higher accuracy than the tested passive system, reporting an overall accuracy of 0.063 mm (SD = 0.025) and 0.259 mm (SD = 0.152), respectively.

Deep learning for image-based liver analysis - A comprehensive review focusing on malignant lesions.

Deep learning-based methods, in particular, convolutional neural networks and fully convolutional networks are now widely used in the medical image analysis domain. The scope of this review focuses on the analysis using deep learning of focal liver lesions, with a special interest in hepatocellular carcinoma and metastatic cancer; and structures like the parenchyma or the vascular system. Here, we address several neural network architectures used for analyzing the anatomical structures and lesions in the liver from various imaging modalities such as computed tomography, magnetic resonance imaging and ultrasound. Image analysis tasks like segmentation, object detection and classification for the liver, liver vessels and liver lesions are discussed. Based on the qualitative search, 91 papers were filtered out for the survey, including journal publications and conference proceedings. The papers reviewed in this work are grouped into eight categories based on the methodologies used. By comparing the evaluation metrics, hybrid models performed better for both the liver and the lesion segmentation tasks, ensemble classifiers performed better for the vessel segmentation tasks and combined approach performed better for both the lesion classification and detection tasks. The performance was measured based on the Dice score for the segmentation, and accuracy for the classification and detection tasks, which are the most commonly used metrics.

Automatic Detection of Aortic Valve Events Using Deep Neural Networks on Cardiac Signals From Epicardially Placed Accelerometer.

BACKGROUND: Miniaturized accelerometers incorporated in pacing leads attached to the myocardium, are used to monitor cardiac function. For this purpose functional indices must be extracted from the acceleration signal. A method that automatically detects the time of aortic valve opening (AVO) and aortic valve closure (AVC) will be helpful for such extraction. We tested if deep learning can be used to detect these valve events from epicardially attached accelerometers, using high fidelity pressure measurements to establish ground truth for these valve events. METHOD: A deep neural network consisting of a CNN, an RNN, and a multi-head attention module was trained and tested on 130 recordings from 19 canines and 159 recordings from 27 porcines covering different interventions. Due to limited data, nested cross-validation was used to assess the accuracy of the method. RESULT: The correct detection rates were 98.9% and 97.1% for AVO and AVC in canines and 98.2% and 96.7% in porcines when defining a correct detection as a prediction closer than 40 ms to the ground truth. The incorrect detection rates were 0.7% and 2.3% for AVO and AVC in canines and 1.1% and 2.3% in porcines. The mean absolute error between correct detections and their ground truth was 8.4 ms and 7.2 ms for AVO and AVC in canines, and 8.9 ms and 10.1 ms in porcines. CONCLUSION: Deep neural networks can be used on signals from epicardially attached accelerometers for robust and accurate detection of the opening and closing of the aortic valve.

Tracking Early Systolic Motion for Assessing Acute Response to Cardiac Resynchronization Therapy in Real Time.

An abnormal systolic motion is frequently observed in patients with left bundle branch block (LBBB), and it has been proposed as a predictor of response to cardiac resynchronization therapy (CRT). Our goal was to investigate if this motion can be monitored with miniaturized sensors feasible for clinical use to identify response to CRT in real time. Motion sensors were attached to the septum and the left ventricular (LV) lateral wall of eighteen anesthetized dogs. Recordings were performed during baseline, after induction of LBBB, and during biventricular pacing. The abnormal contraction pattern in LBBB was quantified by the septal flash index (SFI) equal to the early systolic shortening of the LV septal-to-lateral wall diameter divided by the maximum shortening achieved during ejection. In baseline, with normal electrical activation, there was limited early-systolic shortening and SFI was low (9 ± 8%). After induction of LBBB, this shortening and the SFI significantly increased (88 ± 34%, p < 0.001). Subsequently, CRT reduced it approximately back to baseline values (13 ± 13%, p < 0.001 vs. LBBB). The study showed the feasibility of using miniaturized sensors for continuous monitoring of the abnormal systolic motion of the LV in LBBB and how such sensors can be used to assess response to pacing in real time to guide CRT implantation.

Mixed reality as a novel tool for diagnostic and surgical navigation in orthopaedics.

PURPOSE: This study presents a novel surgical navigation tool developed in mixed reality environment for orthopaedic surgery. Joint and skeletal deformities affect all age groups and greatly reduce the range of motion of the joints. These deformities are notoriously difficult to diagnose and to correct through surgery. METHOD: We have developed a surgical tool which integrates surgical instrument tracking and augmented reality through a head mounted display. This allows the surgeon to visualise bones with the illusion of possessing "X-ray" vision. The studies presented below aim to assess the accuracy of the surgical navigation tool in tracking a location at the tip of the surgical instrument in holographic space. RESULTS: Results show that the average accuracy provided by the navigation tool is around 8 mm, and qualitative assessment by the orthopaedic surgeons provided positive feedback in terms of the capabilities for diagnostic use. CONCLUSIONS: More improvements are necessary for the navigation tool to be accurate enough for surgical applications, however, this new tool has the potential to improve diagnostic accuracy and allow for safer and more precise surgeries, as well as provide for better learning conditions for orthopaedic surgeons in training.

Evaluation of a novel navigation platform for laparoscopic liver surgery with organ deformation compensation using injected fiducials.

In laparoscopic liver resection, surgeons conventionally rely on anatomical landmarks detected through a laparoscope, preoperative volumetric images and laparoscopic ultrasound to compensate for the challenges of minimally invasive access. Image guidance using optical tracking and registration procedures is a promising tool, although often undermined by its inaccuracy. This study evaluates a novel surgical navigation solution that can compensate for liver deformations using an accurate and effective registration method. The proposed solution relies on a robotic C-arm to perform registration to preoperative CT/MRI image data and allows for intraoperative updates during resection using fluoroscopic images. Navigation is offered both as a 3D liver model with real-time instrument visualization, as well as an augmented reality overlay on the laparoscope camera view. Testing was conducted through a pre-clinical trial which included four porcine models. Accuracy of the navigation system was measured through two evaluation methods: liver surface fiducials reprojection and a comparison between planned and navigated resection margins. Target Registration Error with the fiducials evaluation shows that the accuracy in the vicinity of the lesion was 3.78±1.89 mm. Resection margin evaluations resulted in an overall median accuracy of 4.44 mm with a maximum error of 9.75 mm over the four subjects. The presented solution is accurate enough to be potentially clinically beneficial for surgical guidance in laparoscopic liver surgery.

Influence of sampling accuracy on augmented reality for laparoscopic image-guided surgery.

PURPOSE: This study aims to evaluate the accuracy of point-based registration (PBR) when used for augmented reality (AR) in laparoscopic liver resection surgery. MATERIAL AND METHODS: The study was conducted in three different scenarios in which the accuracy of sampling targets for PBR decreases: using an assessment phantom with machined divot holes, a patient-specific liver phantom with markers visible in computed tomography (CT) scans and in vivo, relying on the surgeon's anatomical understanding to perform annotations. Target registration error (TRE) and fiducial registration error (FRE) were computed using five randomly selected positions for image-to-patient registration. RESULTS: AR with intra-operative CT scanning showed a mean TRE of 6.9 mm for the machined phantom, 7.9 mm for the patient-specific phantom and 13.4 mm in the in vivo study. CONCLUSIONS: AR showed an increase in both TRE and FRE throughout the experimental studies, proving that AR is not robust to the sampling accuracy of the targets used to compute image-to-patient registration. Moreover, an influence of the size of the volume to be register was observed. Hence, it is advisable to reduce both errors due to annotations and the size of registration volumes, which can cause large errors in AR systems.