Clinical Application and Further Development of Augmented Reality Guidance for the Surgical Localization of Pediatric Chest Wall Tumors.

BACKGROUND: Surgical treatment of pediatric chest wall tumors requires accurate surgical planning and tumor localization to achieve radical resections while sparing as much healthy tissue as possible. Augmented Reality (AR) could facilitate surgical decision making by improving anatomical understanding and intraoperative tumor localization. We present our clinical experience with the use of an AR system for intraoperative tumor localization during chest wall resections. Furthermore, we present the pre-clinical results of a new registration method to improve our conventional AR system. METHODS: From January 2021, we used the HoloLens 2 for pre-incisional tumor localization during all chest wall resections inside our center. A patient-specific 3D model was projected onto the patient by use of a five-point registration method based on anatomical landmarks. Furthermore, we developed and pre-clinically tested a surface matching method to allow post-incisional AR guidance by performing registration on the exposed surface of the ribs. RESULTS: Successful registration and holographic overlay were achieved in eight patients. The projection seemed most accurate when landmarks were positioned in a non-symmetric configuration in proximity to the tumor. Disagreements between the overlay and expected tumor location were mainly due to user-dependent registration errors. The pre-clinical tests of the surface matching method proved the feasibility of registration on the exposed ribs. CONCLUSIONS: Our results prove the applicability of AR guidance for the pre- and post-incisional localization of pediatric chest wall tumors during surgery. The system has the potential to enable intraoperative 3D visualization, hereby facilitating surgical planning and management of chest wall resections. LEVEL OF EVIDENCE: IV TYPE OF STUDY: Treatment Study.

Fully automatic brain tumor segmentation for 3D evaluation in augmented reality.

OBJECTIVE: For currently available augmented reality workflows, 3D models need to be created with manual or semiautomatic segmentation, which is a time-consuming process. The authors created an automatic segmentation algorithm that generates 3D models of skin, brain, ventricles, and contrast-enhancing tumor from a single T1-weighted MR sequence and embedded this model into an automatic workflow for 3D evaluation of anatomical structures with augmented reality in a cloud environment. In this study, the authors validate the accuracy and efficiency of this automatic segmentation algorithm for brain tumors and compared it with a manually segmented ground truth set. METHODS: Fifty contrast-enhanced T1-weighted sequences of patients with contrast-enhancing lesions measuring at least 5 cm3 were included. All slices of the ground truth set were manually segmented. The same scans were subsequently run in the cloud environment for automatic segmentation. Segmentation times were recorded. The accuracy of the algorithm was compared with that of manual segmentation and evaluated in terms of Sørensen-Dice similarity coefficient (DSC), average symmetric surface distance (ASSD), and 95th percentile of Hausdorff distance (HD95). RESULTS: The mean ± SD computation time of the automatic segmentation algorithm was 753 ± 128 seconds. The mean ± SD DSC was 0.868 ± 0.07, ASSD was 1.31 ± 0.63 mm, and HD95 was 4.80 ± 3.18 mm. Meningioma (mean 0.89 and median 0.92) showed greater DSC than metastasis (mean 0.84 and median 0.85). Automatic segmentation had greater accuracy for measuring DSC (mean 0.86 and median 0.87) and HD95 (mean 3.62 mm and median 3.11 mm) of supratentorial metastasis than those of infratentorial metastasis (mean 0.82 and median 0.81 for DSC; mean 5.26 mm and median 4.72 mm for HD95). CONCLUSIONS: The automatic cloud-based segmentation algorithm is reliable, accurate, and fast enough to aid neurosurgeons in everyday clinical practice by providing 3D augmented reality visualization of contrast-enhancing intracranial lesions measuring at least 5 cm3. The next steps involve incorporation of other sequences and improving accuracy with 3D fine-tuning in order to expand the scope of augmented reality workflow.

Holographic Augmented Reality for DIEP Flap Harvest.

BACKGROUND: During a deep inferior epigastric perforator (DIEP) flap harvest, the identification and localization of the epigastric arteries and its perforators are crucial. Holographic augmented reality is an innovative technique that can be used to visualize this patient-specific anatomy extracted from a computed tomographic scan directly on the patient. This study describes an innovative workflow to achieve this. METHODS: A software application for the Microsoft HoloLens was developed to visualize the anatomy as a hologram. By using abdominal nevi as natural landmarks, the anatomy hologram is registered to the patient. To ensure that the anatomy hologram remains correctly positioned when the patient or the user moves, real-time patient tracking is obtained with a quick response marker attached to the patient. RESULTS: Holographic augmented reality can be used to visualize the epigastric arteries and its perforators in preparation for a deep inferior epigastric perforator flap harvest. CONCLUSIONS: Potentially, this workflow can be used visualize the vessels intraoperatively. Furthermore, this workflow is intuitive to use and could be applied for other flaps or other types of surgery.

Three-dimensional facial volume analysis using algorithm-based personalized aesthetic templates.

Three-dimensional stereophotogrammetry is commonly used to assess volumetric changes after facial procedures. A lack of clear landmarks in aesthetic regions complicates the reproduction of selected areas in sequential images. A three-dimensional volumetric analysis was developed based on a personalized aesthetic template. The accuracy and reproducibility of this method were assessed. Six female volunteers were photographed using the 3dMDtrio system according to a clinical protocol, twice at baseline (T1) and twice after 1year (T2). A styrofoam head was used as control. A standardized aesthetic template was morphed over the baseline images of the volunteers using a coherent point drift algorithm. The resulting personalized template was projected over all sequential images to assess surface area differences, volume differences, and root mean square errors. In 12 well-defined aesthetic areas, mean average surface area and volume differences between the two T1 images ranged from -7.6mm2 to 10.1mm2 and -0.11cm3 to 0.13cm3, respectively. T1 root mean square errors ranged between 0.24mm and 0.62mm (standard deviation 0.18-0.73mm). Comparable differences were found between the T2 images. An increase in volume between T1 and T2 was only observed for volunteers who gained in body weight. Personalized aesthetic templates are an accurate and reproducible method to assess changes in aesthetic areas.

Three-dimensional facial development of children with unilateral cleft lip and palate during the first year of life in comparison with normative average faces.

BACKGROUND: Stereophotogrammetry can be used to study facial morphology in both healthy individuals as well as subjects with orofacial clefts because it shows good reliability, ability to capture images rapidly, archival capabilities, and high resolution, and does not require ionizing radiation. This study aimed to compare the three-dimensional (3D) facial morphology of infants born with unilateral cleft lip and palate (UCLP) with an age-matched normative 3D average face before and after primary closure of the lip and soft palate. METHODS: Thirty infants with a non-syndromic complete unilateral cleft lip, alveolus, and palate participated in the study. Three-dimensional images were acquired at 3, 6, 9, and 12 months of age. All subjects were treated according to the primary surgical protocol consisting of surgical closure of the lip and the soft palate at 6 months of age. Three-dimensional images of UCLP patients at 3, 6 (pre-treatment), 9, and 12 months of age were superimposed on normative datasets of average facial morphology using the children's reference frame. Distance maps of the complete 3D facial surface and the nose, upper lip, chin, forehead, and cheek regions were developed. RESULTS: Assessments of the facial morphology of UCLP and control subjects by using color-distance maps showed large differences in the upper lip region at the location of the cleft defect and an asymmetry at the nostrils at 3 and 6 months of age. At 9 months of age, the labial symmetry was completely restored although the tip of the nose towards the unaffected side showed some remnant asymmetry. At 12 months of age, the symmetry of the nose improved, with only some remnant asymmetry noted on both sides of the nasal tip. At all ages, the mandibular and chin regions of the UCLP patients were 2.5-5 mm posterior to those in the average controls. CONCLUSION: In patients with UCLP deviations from the normative average 3D facial morphology of age-matched control subjects existed for the upper lip, nose, and even the forehead before lip and soft palate closure was performed. Compared to the controls symmetry in the upper lip was restored, and the shape of the upper lip showed less variation after primary lip and soft palate closure. At this early age, retrusion of the soft-tissue mandible and chin, however, seems to be developing already.

Uniform 3D meshes to establish normative facial averages of healthy infants during the first year of life.

Three-dimensional (3D) surface imaging systems are replacing direct anthropometry as the preferred method for capturing facial soft-tissues. Aims of this study were: (1) to develop normative average 3D faces of healthy infants aged 3, 6, 9, and 12 months and (2) to describe normative average 3D facial growth data in infants aged 3 to 12 months. Three-dimensional images of 50 healthy children were acquired at 3, 6, 9, and 12 months of age using the 3dMDcranial system. Four average faces with uniform meshes (3, 6, 9, and 12 months) were developed and registered based on the children's reference frames. Distance maps of growth of the total facial surface and of the nose, upper lip, chin, forehead and cheeks for the intervals 3 to 6 months, 6 to 9 months, and 9 to 12 months of age were calculated. Mean growth of the total facial surface was 3.9 mm (standard deviation [SD] 1.2 mm), 3.5 mm (SD 0.9 mm), and 1.6 mm (SD 0.7 mm) at 3 to 6 months, 6 to 9 months, and 9 to 12 months, respectively. Regarding the selected regions of the face, the mean growth of the nose and upper lip were the largest (3.7 mm and 3.6 mm, respectively) between 6 and 9 months of age. The mean growth of the forehead, cheeks and chin were the largest (5.4 mm, 3.2, and 4.7 mm, respectively) between 3 and 6 months of age. For all facial regions, growth clearly diminished from 9 to 12 months of age. Normative data on the growth of the full face, nose, upper lip, chin, forehead and cheeks are presented. Such data can be used in future studies to identify the effectiveness of treatment of orofacial deformities such as orofacial clefts during the first year of life.

Toward Holographic-Guided Surgery.

The implementation of augmented reality (AR) in image-guided surgery (IGS) can improve surgical interventions by presenting the image data directly on the patient at the correct position and in the actual orientation. This approach can resolve the switching focus problem, which occurs in conventional IGS systems when the surgeon has to look away from the operation field to consult the image data on a 2-dimensional screen. The Microsoft HoloLens, a head-mounted AR display, was combined with an optical navigation system to create an AR-based IGS system. Experiments were performed on a phantom model to determine the accuracy of the complete system and to evaluate the effect of adding AR. The results demonstrated a mean Euclidean distance of 2.3 mm with a maximum error of 3.5 mm for the complete system. Adding AR visualization to a conventional system increased the mean error by 1.6 mm. The introduction of AR in IGS was promising. The presented system provided a solution for the switching focus problem and created a more intuitive guidance system. With a further reduction in the error and more research to optimize the visualization, many surgical applications could benefit from the advantages of AR guidance.

Bone-borne surgically assisted rapid maxillary expansion: A retrospective three-dimensional evaluation of the asymmetry in expansion.

PURPOSE: Asymmetrical expansion occurs in patients treated with Surgically Assisted Rapid Maxillary Expansion (SARME). In the clinical setting, this asymmetrical expansion is seen in multiple directions. However, the frequency, actual directions and amount of asymmetry are unclear. Hence, the aim of this study was to analyze the directions and amount of asymmetrical lateral expansion in non-syndromic patients with transversal maxillary hypoplasia on employing bone-borne transpalatal distraction by means of SARME. Treatment involved corticotomies of all four bony supports, including pterygomaxillary disjunction. MATERIALS AND METHODS: A retrospective case series was formed from patients treated with SARME. Pre- and postdistraction Cone Beam Computed Tomography scans were superimposed. A reference frame was created to analyze lateral expansion asymmetries in five directions. RESULTS: Clinical relevant asymmetries (>3.0 mm) in at least one of the investigated directions occurred in 55% of the patients. Lateral expansion asymmetries occurred mostly in the inferior-anterior part between the left and right segment and asymmetry in total expansion was noted between the anterior and posterior part of the maxilla. CONCLUSION: This study confirms the clinical suspicion that using SARME with a bone-borne distractor and pterygomaxillary disjunction to treat non-syndromic patients with transversal maxillary hypoplasia, results in regular asymmetrical lateral expansion.