Virtual, mixed, and augmented realities: A commentary on their significance in cognitive neuroscience and neuropsychology.

The integration of virtual, mixed, and augmented reality technologies in cognitive neuroscience and neuropsychology represents a transformative frontier. In this Commentary, we conducted a meta-analysis of studies that explored the impact of Virtual Reality (VR), Mixed Reality (MR), and Augmented Reality (AR) on cognitive neuroscience and neuropsychology. Our review highlights the versatile applications of VR, ranging from spatial cognition assessments to rehabilitation for Traumatic Brain Injury. We found that MR and AR offer innovative avenues for cognitive training, particularly in memory-related disorders. The applications extend to addressing social cognition disorders and serving as therapeutic interventions for mental health issues. Collaborative efforts between neuroscientists and technology developers are crucial, with reinforcement learning and neuroimaging studies enhancing the potential for improved outcomes. Ethical considerations, including informed consent, privacy, and accessibility, demand careful attention. Our review identified common aspects of the meta-analysis, including the potential of VR technologies in cognitive neuroscience and neuropsychology, the use of MR and AR in memory research, and the role of VR in neurorehabilitation and therapy.

The application of extended reality technology-assisted intraoperative navigation in orthopedic surgery.

Extended reality (XR) technology refers to any situation where real-world objects are enhanced with computer technology, including virtual reality, augmented reality, and mixed reality. Augmented reality and mixed reality technologies have been widely applied in orthopedic clinical practice, including in teaching, preoperative planning, intraoperative navigation, and surgical outcome evaluation. The primary goal of this narrative review is to summarize the effectiveness and superiority of XR-technology-assisted intraoperative navigation in the fields of trauma, joint, spine, and bone tumor surgery, as well as to discuss the current shortcomings in intraoperative navigation applications. We reviewed titles of more than 200 studies obtained from PubMed with the following search terms: extended reality, mixed reality, augmented reality, virtual reality, intraoperative navigation, and orthopedic surgery; of those 200 studies, 69 related papers were selected for abstract review. Finally, the full text of 55 studies was analyzed and reviewed. They were classified into four groups-trauma, joint, spine, and bone tumor surgery-according to their content. Most of studies that we reviewed showed that XR-technology-assisted intraoperative navigation can effectively improve the accuracy of implant placement, such as that of screws and prostheses, reduce postoperative complications caused by inaccurate implantation, facilitate the achievement of tumor-free surgical margins, shorten the surgical duration, reduce radiation exposure for patients and surgeons, minimize further damage caused by the need for visual exposure during surgery, and provide richer and more efficient intraoperative communication, thereby facilitating academic exchange, medical assistance, and the implementation of remote healthcare.

Accessibility and inclusiveness of new information and communication technologies for disabled users and content creators in the Metaverse.

PURPOSE: The study critically reassesses existing Metaverse concepts and proposes a novel framework for inclusiveness of physically disabled artists. The purpose is to enable and inspire physically disabled users and content creators to participate in the evolving concept of the Metaverse. The article also highlights the need for standards and regulations governing the inclusion of people with disabilities in Metaverse projects. MATERIALS AND METHODS: The study examines current information technologies and their relevance to the inclusion of physically disabled individuals in the Metaverse. We analyse existing Metaverse concepts, exploring emerging information technologies such as Virtual and Augmented Reality, and the Internet of Things. The emerging framework in the article is based on the active involvement of disabled creatives in the development of solutions for inclusivity. RESULTS: The review reveals that despite the proliferation of Blockchain Metaverse projects, the inclusion of physically disabled individuals in the Metaverse remains distant, with limited standards and regulations in place. The article proposes a concept of the Metaverse that leverages emerging technologies, to enable greater engagement of disabled creatives. This approach is designed to enhance inclusiveness in the Metaverse landscape. CONCLUSIONS: Active involvement of physically disabled individuals in the design and development of Metaverse platforms is crucial for promoting inclusivity. The framework for accessibility and inclusiveness in decentralised Metaverses provides a basis for the meaningful participation of disabled creatives. The article emphasises the importance of addressing the mechanisms for art production by individuals with disabilities in the emerging Metaverse landscape.IMPLICATIONS FOR REHABILITATIONThis article addresses a global challenge related to helping disabled people operate in the modern society, targeting new and emerging technologies, and enabling early understanding of required actions for inclusiveness of people with disabilities in the Metaverse.The increased use of advanced technologies (e.g., AI and IoT) in the Metaverse, amplified the importance of this research being conducted.The aggregate impact from this research for science and society is a more inclusive, and unbiassed Metaverses that are compliant with regulations on anti-disability discrimination. This is followed by the secondary values, related to increased technological opportunities from a breakthrough in designing new, more inclusive, and autonomous devices.The research study presents a new framework for integrating new technologies in existing Metaverses, resulting with a stronger accessibility and inclusiveness of the Metaverse. This creates a new understanding on how new technologies can be used for disability discrimination prevention and early understanding of disability requirements. We also highlighted normative constraints and the need for further reflection and weighing to avoid dystopian futures for the physically disabled in relation to the Metaverse.

A novel mixed reality system to manage phantom pain in-home: results of a pilot clinical trial.

Introduction: Mirror therapy for phantom limb pain (PLP) is a well-accepted treatment method that allows participants to use a mirror to visually perceive the missing limb. Mixed reality options are now becoming increasingly available, but an in-home virtual mirror therapy option has yet to be adequately investigated. Methods: We had previously developed a mixed reality system for Managing Phantom Pain (Mr. MAPP) that registers the intact limb and mirrors it onto the amputated limb with the system's visual field, allowing the user to engage with interactive games targeting different large lower limb movements. Feasibility and pilot outcomes of treating patients with lower extremity PLP by using Mr. MAPP at home for 1 month were evaluated in this study. Pain intensity and interference were assessed using the McGill Pain Questionnaire, Brief Pain Inventory, and a daily exercise diary. Function was assessed using the Patient Specific Functional Scale (PSFS). The clinical trial registry number for this study is NCT04529083. Results: This pilot study showed that it was feasible for patients with PLP to use Mr. MAPP at home. Among pilot clinical outcomes, statistically significant differences were noted in mean current pain intensity [1.75 (SD = 0.46) to 1.125 (SD = 0.35) out of 5, P = .011] and PSFS goal scores [4.28 (SD = 2.27) to 6.22 (SD = 2.58) out of 10, P = .006], with other outcome measures showing non-significant trends towards improvement. Discussion: This pilot study revealed that in-home use of Mr. MAPP has potential to provide pain relief and improve function in patients with lower extremity PLP and is feasible. Each scale used provided unique perspective on the functional impact of PLP. Further expanded studies and investigation, including a fully powered clinical trial, with these scales are warranted. Clinical Trial Registration: https://www.clinicaltrials.gov/ct2/show/NCT04529083, Identifier: NCT04529083.

The Role of Augmented Reality in the Advancement of Minimally Invasive Surgery Procedures: A Scoping Review.

The purpose of this review was to analyze the evidence on the role of augmented reality (AR) in the improvement of minimally invasive surgical (MIS) procedures. A scoping literature search of the PubMed and ScienceDirect databases was performed to identify articles published in the last five years that addressed the direct impact of AR technology on MIS procedures or that addressed an area of education or clinical care that could potentially be used for MIS development. A total of 359 studies were screened and 31 articles were reviewed in depth and categorized into three main groups: Navigation, education and training, and user-environment interfaces. A comparison of studies within the different application groups showed that AR technology can be useful in various disciplines to advance the development of MIS. Although AR-guided navigation systems do not yet offer a precision advantage, benefits include improved ergonomics and visualization, as well as reduced surgical time and blood loss. Benefits can also be seen in improved education and training conditions and improved user-environment interfaces that can indirectly influence MIS procedures. However, there are still technical challenges that need to be addressed to demonstrate added value to patient care and should be evaluated in clinical trials with sufficient patient numbers or even in systematic reviews or meta-analyses.

A new preoperative localization of pulmonary nodules guided by mixed reality: a pilot study of an animal model.

Background: With the popularity of high-resolution computed tomography (HRCT), more and more pulmonary nodules are being discovered. Video-assisted thoracoscopic surgery (VATS) has become the first choice for surgical treatment of pulmonary nodules. The use of accurate preoperative localization is crucial for successful resection in VATS. At present, there are many kinds of preoperative localization methods, but there are certain disadvantages. This study aimed to evaluate the feasibility and safety of mixed reality (MR)-guided pulmonary nodules localization, which is a new method that can benefit patients to a greater extent. Methods: By constructing an animal model of pulmonary nodules localization, 28 cases of pulmonary nodules were located by MR-guided localization. We recorded the localization accuracy, localization time, insertion attempts, and incidence of complications related to localization under MR-guidance. Results: All 28 nodules were successfully located: the deviation of MR-guided localization was 5.71±2.59 mm, localization time was 8.07±1.44 min, and insertion attempts was 1. A pneumothorax and localizer dislodgement occurred in 1 case, respectively. Conclusions: Since preoperative localization is critical for VATS resection of pulmonary nodules, we investigated a new localization method. As indicated by our study, MR-guided localization of pulmonary nodules is feasible and safe, which is worthy of further research and promotion. We have also registered corresponding clinical trials to further investigate and help to improve our understanding of this technique.

Mixed reality for extraction of maxillary mesiodens.

BACKGROUND: Mesiodentes are the most common supernumerary teeth. The cause is not fully understood, although proliferations of genetic factors and the dental lamina have been implicated. Mesiodentes can cause delayed or ectopic eruption of permanent incisors, which can further alter occlusion and appearance. Careful attention should be paid to the position and direction of the mesiodentes because of possible damage to adjacent roots in the permanent dentition period, errant extraction in the deciduous and mixed dentition periods, and damage to the permanent tooth embryo. To avoid these complications, we applied mixed reality (MR) technology using the HoloLens® (Microsoft, California). In this study, we report on three cases of mesiodentes extraction under general anesthesia using MR technology. RESULTS: The patients ranged in age from 6 to 11 years, all three were boys, and the direction of eruption was inverted in all cases. The extraction approach was palatal in two cases and labial in one case. The average operative time was 32 min, and bleeding was minimal in all cases. No intraoperative or postoperative complications occurred. An image was shared preoperatively with all the surgeons using an actual situation model. Three surgeons used Microsoft HoloLens® during surgery, shared MR, and operated while superimposing the application image in the surgical field. CONCLUSIONS: The procedure was performed safely; further development of MR surgery support systems in the future is suggested.

Augmented reality (AR) in minimally invasive surgery (MIS) training: where are we now in Italy? The Italian Society of Endoscopic Surgery (SICE) ARMIS survey.

Minimally invasive surgery (MIS) is a widespread approach in general surgery. Computer guiding software, such as the augmented reality (AR), the virtual reality (VR) and mixed reality (MR), has been proposed to help surgeons during MIS. This study aims to report these technologies' current knowledge and diffusion during surgical training in Italy. A web-based survey was developed under the aegis of the Italian Society of Endoscopic Surgery (SICE). Two hundred and seventeen medical doctors' answers were analyzed. Participants were surgeons (138, 63.6%) and residents in surgery (79, 36.4%). The mean knowledge of the role of the VR, AR and MR in surgery was 4.9 ± 2.4 (range 1-10). Most of the participants (122, 56.2%) did not have experience with any proposed technologies. However, although the lack of experience in this field, the answers about the functioning of the technologies were correct in most cases. Most of the participants answered that VR, AR and MR should be used more frequently for the teaching and training and during the clinical activity (170, 80.3%) and that such technologies would make a significant contribution, especially in training (183, 84.3%) and didactic (156, 71.9%). Finally, the main limitations to the diffusion of these technologies were the insufficient knowledge (182, 83.9%) and costs (175, 80.6%). Based on the present study, in Italy, the knowledge and dissemination of these technologies are still limited. Further studies are required to establish the usefulness of AR, VR and MR in surgical training.

Using the DiCoT framework for integrated multimodal analysis in mixed-reality training environments.

Simulation-based training (SBT) programs are commonly employed by organizations to train individuals and teams for effective workplace cognitive and psychomotor skills in a broad range of applications. Distributed cognition has become a popular cognitive framework for the design and evaluation of these SBT environments, with structured methodologies such as Distributed Cognition for Teamwork (DiCoT) used for analysis. However, the analysis and evaluations generated by such distributed cognition frameworks require extensive domain-knowledge and manual coding and interpretation, and the analysis is primarily qualitative. In this work, we propose and develop the application of multimodal learning analysis techniques to SBT scenarios. Using these analysis methods, we can use the rich multimodal data collected in SBT environments to generate more automated interpretations of trainee performance that supplement and extend traditional DiCoT analysis. To demonstrate the use of these methods, we present a case study of nurses training in a mixed-reality manikin-based (MRMB) training environment. We show how the combined analysis of the video, speech, and eye-tracking data collected as the nurses train in the MRMB environment supports and enhances traditional qualitative DiCoT analysis. By applying such quantitative data-driven analysis methods, we can better analyze trainee activities online in SBT and MRMB environments. With continued development, these analysis methods could be used to provide targeted feedback to learners, a detailed review of training performance to the instructors, and data-driven evidence for improving the environment to simulation designers.

Applications of Mixed Reality Technology in Orthopedics Surgery: A Pilot Study.

Objective: The aim of this study is to explore the potential of mixed reality (MR) technology in the visualization of orthopedic surgery. Methods: The visualization system with MR technology is widely used in orthopedic surgery. The system is composed of a 3D imaging workstation, a cloud platform, and an MR space station. An intelligent segmentation algorithm is adopted on the 3D imaging workstation to create a 3D anatomical model with zooming and rotation effects. This model is then exploited for efficient 3D reconstruction of data for computerized tomography (CT) and magnetic resonance imaging (MRI). Additionally, the model can be uploaded to the cloud platform for physical parameter tuning, model positioning, rendering and high-dimensional display. Using Microsoft's HoloLens glasses in combination with the MR system, we project and view 3D holograms in real time under different clinical scenarios. After each procedure, nine surgeons completed a Likert-scale questionnaire on communication and understanding, spatial awareness and effectiveness of MR technology use. In addition to that, the National Aeronautics and Space Administration Task Load Index (NASA-TLX) is also used to evaluate the workload of MR hologram support. Results: 1) MR holograms can clearly show the 3D structures of bone fractures, which improves the understanding of different fracture types and the design of treatment plans; 2) Holograms with three-dimensional lifelike dynamic features provide an intuitive communication tool among doctors and also between doctors and patients; 3) During surgeries, a full lesion hologram can be obtained and blended in real time with a patient's virtual 3D digital model in order to give surgeons superior visual guidance through novel high-dimensional "perspectives" of the surgical area; 4) Hologram-based magnetic navigation improves the accuracy and safety of the screw placement in orthopaedics surgeries; 5) The combination of mixed reality cloud platform and telemedicine system based on 5G provides a new technology platform for telesurgery collaboration. Results of qualitative study encourage the usage of MR technology for orthopaedics surgery. Analysis of the Likert-scale questionnaire shows that MR adds significant value to understanding and communication, spatial awareness, learning and effectiveness. Based on the NASA TLX-scale questionnaire results, mixed reality scored significantly lower under the "mental," "temporal," "performance," and "frustration" categories compared to usual 2D. Conclusion: The integration of MR technology in orthopaedic surgery reduces the dependence on surgeons' experience and provides personalized 3D visualization models for accurate diagnosis and treatment of orthopaedic abnormalities. This integration is clearly one of the prominent future development directions in medical surgery.

Mixed reality and three dimensional printed models for resection of maxillary tumor: a case report.

In the field of oral and maxillofacial surgery, many institutions have recently begun using three-dimensional printers to create three-dimensional models and mixed reality in a variety of diseases. Here, we report the actual situation model which we made using three-dimensional printer from virtual operation data and the resection that was performed while grasping a maxillary benign tumor and neighboring three-dimensional structure by designing an application for Microsoft® HoloLens, and using Mixed Reality surgery support during the procedure.

Towards Aircraft Maintenance Metaverse Using Speech Interactions with Virtual Objects in Mixed Reality.

Metaverses embedded in our lives create virtual experiences inside of the physical world. Moving towards metaverses in aircraft maintenance, mixed reality (MR) creates enormous opportunities for the interaction with virtual airplanes (digital twin) that deliver a near-real experience, keeping physical distancing during pandemics. 3D twins of modern machines exported to MR can be easily manipulated, shared, and updated, which creates colossal benefits for aviation colleges who still exploit retired models for practicing. Therefore, we propose mixed reality education and training of aircraft maintenance for Boeing 737 in smart glasses, enhanced with a deep learning speech interaction module for trainee engineers to control virtual assets and workflow using speech commands, enabling them to operate with both hands. With the use of the convolutional neural network (CNN) architecture for audio features and learning and classification parts for commands and language identification, the speech module handles intermixed requests in English and Korean languages, giving corresponding feedback. Evaluation with test data showed high accuracy of prediction, having on average 95.7% and 99.6% on the F1-Score metric for command and language prediction, respectively. The proposed speech interaction module in the aircraft maintenance metaverse further improved education and training, giving intuitive and efficient control over the operation, enhancing interaction with virtual objects in mixed reality.

Current innovation in virtual and augmented reality in spine surgery.

In spinal surgery, outcomes are directly related both to patient and procedure selection, as well as the accuracy and precision of instrumentation placed. Poorly placed instrumentation can lead to spinal cord, nerve root or vascular injury. Traditionally, spine surgery was performed by open methods and placement of instrumentation under direct visualization. However, minimally invasive surgery (MIS) has seen substantial advances in spine, with an ever-increasing range of indications and procedures. For these reasons, novel methods to visualize anatomy and precisely guide surgery, such as intraoperative navigation, are extremely useful in this field. In this review, we present the recent advances and innovations utilizing simulation methods in spine surgery. The application of these techniques is still relatively new, however quickly being integrated in and outside the operating room. These include virtual reality (VR) (where the entire simulation is virtual), mixed reality (MR) (a combination of virtual and physical components), and augmented reality (AR) (the superimposition of a virtual component onto physical reality). VR and MR have primarily found applications in a teaching and preparatory role, while AR is mainly applied in hands-on surgical settings. The present review attempts to provide an overview of the latest advances and applications of these methods in the neurosurgical spine setting.

A review on the applications of virtual reality, augmented reality and mixed reality in surgical simulation: an extension to different kinds of surgery.

Background: Research proves that the apprenticeship model, which is the gold standard for training surgical residents, is obsolete. For that reason, there is a continuing effort toward the development of high-fidelity surgical simulators to replace the apprenticeship model. Applying Virtual Reality Augmented Reality (AR) and Mixed Reality (MR) in surgical simulators increases the fidelity, level of immersion and overall experience of these simulators.Areas covered: The objective of this review is to provide a comprehensive overview of the application of VR, AR and MR for distinct surgical disciplines, including maxillofacial surgery and neurosurgery. The current developments in these areas, as well as potential future directions, are discussed.Expert opinion: The key components for incorporating VR into surgical simulators are visual and haptic rendering. These components ensure that the user is completely immersed in the virtual environment and can interact in the same way as in the physical world. The key components for the application of AR and MR into surgical simulators include the tracking system as well as the visual rendering. The advantages of these surgical simulators are the ability to perform user evaluations and increase the training frequency of surgical residents.

[Application of mixed reality technique for the surgery of oral and maxillofacial tumors].

OBJECTIVE: To explore the application of mixed reality technique for the surgery of oral and maxillofacial tumors. METHODS: In this study, patients with a diagnosis of an oral and maxillofacial tumor who were referred to Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology from December 2018 to January 2020 were selected. The preoperative contrast-enhanced computed tomography data of the patients were imported into StarAtlas Holographic Medical Imaging System (Visual 3D Corp., Beijing, China). Three-dimensional (3D) model of tumor and key structures, such as skeleton and vessels were reconstructed to three-dimensionally present the spatial relationship between them, followed with the key structures delineation and preoperative virtual surgical planning. By using mixed reality technique, the real-time 3D model was displayed stereotactically in the surgical site. While keeping sterile during operation, the surgeon could use simple gestures to adjust the 3D model, and observed the location, range, and size of tumor and the key structures adjacent to the tumor. Mixed reality technique was used to assist the operation: 3D model registration was performed for guidance before tumor excision; intraoperative real-time verification was performed during tumor exposure and after excision of the tumor. The Likert scale was used to evaluate the application of mixed reality technique after the operation. RESULTS: Eight patients underwent mixed reality assisted tumor resection, and all of them successfully completed the operation. The average time of the 3D model registration was 12.0 minutes. In all the cases, the surgeon could intuitively and three-dimensionally observe the 3D model of the tumor and the surrounding anatomical structures, and could adjust the model during the operation. The results of the Likert scale showed that mixed reality technique got high scores in terms of perceptual accuracy, helping to locate the anatomical parts, the role of model guidance during surgery, and the potential for improving surgical safety (4.22, 4.19, 4.16, and 4.28 points respectively). Eight patients healed well without perioperative complications. CONCLUSION: By providing real-time stereotactic visualization of anatomy of surgical site and guiding the operation process through 3D model, mixed reality technique could improve the accuracy and safety of the excision of oral and maxillofacial tumors.

Application of mixed reality technique for the surgery of oral and maxillofacial tumors / 北京大学学报(医学版)

OBJECTIVE@#To explore the application of mixed reality technique for the surgery of oral and maxillofacial tumors.@*METHODS@#In this study, patients with a diagnosis of an oral and maxillofacial tumor who were referred to Department of Oral and Maxillofacial Surgery, Peking University School and Hospital of Stomatology from December 2018 to January 2020 were selected. The preoperative contrast-enhanced computed tomography data of the patients were imported into StarAtlas Holographic Medical Imaging System (Visual 3D Corp., Beijing, China). Three-dimensional (3D) model of tumor and key structures, such as skeleton and vessels were reconstructed to three-dimensionally present the spatial relationship between them, followed with the key structures delineation and preoperative virtual surgical planning. By using mixed reality technique, the real-time 3D model was displayed stereotactically in the surgical site. While keeping sterile during operation, the surgeon could use simple gestures to adjust the 3D model, and observed the location, range, and size of tumor and the key structures adjacent to the tumor. Mixed reality technique was used to assist the operation: 3D model registration was performed for guidance before tumor excision; intraoperative real-time verification was performed during tumor exposure and after excision of the tumor. The Likert scale was used to evaluate the application of mixed reality technique after the operation.@*RESULTS@#Eight patients underwent mixed reality assisted tumor resection, and all of them successfully completed the operation. The average time of the 3D model registration was 12.0 minutes. In all the cases, the surgeon could intuitively and three-dimensionally observe the 3D model of the tumor and the surrounding anatomical structures, and could adjust the model during the operation. The results of the Likert scale showed that mixed reality technique got high scores in terms of perceptual accuracy, helping to locate the anatomical parts, the role of model guidance during surgery, and the potential for improving surgical safety (4.22, 4.19, 4.16, and 4.28 points respectively). Eight patients healed well without perioperative complications.@*CONCLUSION@#By providing real-time stereotactic visualization of anatomy of surgical site and guiding the operation process through 3D model, mixed reality technique could improve the accuracy and safety of the excision of oral and maxillofacial tumors.

[Application of mixed reality real-time navigation combined with 3D visualization in complicated hepatectomy].

Objective: To explore the application value of mixed-reality (MR) navigation combined with three-dimensional visualization technique in complicated hepatectomy. Methods: A retrospective analysis of the clinical data of fifty patients with complex liver cancer who underwent liver resection in the Department of hepatobiliary surgery of Nantong Tumor Hospital during September 2015 to October 2017 was conducted. These patients were randomly divided into control group and experimental group, including 22 cases in the experimental group, using three-dimensional visualization technique for preoperative assessment, and MR surgery intraoperative navigation technology, and 28 cases in the control group, in which the three-dimensional visualization technique and the MR navigation were not performed. The surgeons of the two groups were the same. The preoperative, intraoperative and postoperative indexes of the two groups were compared and analyzed. Results: There was no significant difference in preoperative indexes. But in the intraoperative indexes, the operation time [(82.3±10.4) min vs (96.6±22.7) min] and hepatic portal blocking time [(12.2±3.8) min VS (15.8±4.2) min] of the experimental group were significantly shorter than those of the control group, meanwhile, the amount of intraoperative bleeding [(486.4±118.5) ml vs (567.2±142.8) ml] and the volume of intraoperative blood transfusion of [(1.8±0.2) U vs (2.5±0.6) U] were significantly decreased. These differences were statistically significant. In the postoperatively, the total incidence of postoperative complications (4/22 vs 13/28) of the experimental group was lower than that of the control group, and the difference was statistically significant. Conclusion: In complicated hepatectomy, MR intraoperative navigation combined with three-dimensional visualization technique which used for preoperative assessment can significantly shorten operation time, hepatic portal blocking time, significantly reduce intraoperative blood loss and transfusion volume, and significantly reduce the incidence of postoperative total complications, which is of clinical value.

Application and exploration of mixed reality technology in accurate operation of percutaneous vertebroplasty of spine / 上海交通大学学报（医学版）

Objective • To explore the application and clinical significance of mixed reality technology in percutaneous vertebroplasty of spine. Methods • A 79-year-old female patient with multiple osteoporotic compression fractures was selected in Department of Orthopedics, Shanghai General Hospital, Shanghai Jiao Tong University School of Medicine. Lumbar 64-row CT scan was performed to obtain digital imaging and communications in medicine (DICOM) data. Three-dimensional reconstruction software was imported to construct the model of spinal compression fracture. Visual edge detection, segmentation, extraction and calculation of spinal skeleton were performed to generate a three-dimensional network. Collateral model, by mixed reality smartglasses,three-dimensional image of the compression spine image was compared to the real world. Mixed reality technology could assist the surgeon to complete the preoperative simulation, communicate effectively with the patient, and guide percutaneous vertebroplasty of lumber. Results • With mixed reality, the puncture was accurate. The distribution of bone cement was good. The pain of patients was relieved obviously after operation. By mixed reality, the operation time was shortened and the side-effects were reduced. Based on reconstruction of mixed reality technology from original CT imaging data, the compression fracture and anatomical structure were visualized clearly during operation. The operation path was clear and the compression fracture forming site was accurately located. Conclusion •Mixed reality technology is helpful for accurate puncture in percutaneous vertebroplasty of spine, and has potential for development in minimally invasive treatment of spine.