Intra-operative augmented reality in distal locking.

PURPOSE: To design an augmented reality solution that assists surgeons during the distal locking of intramedullary nailing procedures. METHOD: Traditionally, the procedure is performed under X-ray guidance and requires a significant amount of time and radiation exposure. To absolve these complications, we propose video guidance that allows surgeons to achieve both the down-the-beam position of the intramedullary nail and its subsequent locking. For the down-the-beam position, the IM nail pose in X-ray is calculated using a 2D/3D registration scheme and later related to the patient leg pose which is calculated using video-tracked AR markers. For the distal locking, surgeons use an augmented radiolucent drill in which its tip position is detected and tracked in real-time under video guidance. VALIDATION: To evaluate the feasibility of our solution, we performed a preclinical study on dry bone phantom with the participation of four clinicians. RESULTS: Participants achieved 100 % success rate in the down-the beam positioning and 93 % success rate in distal locking using only two X-ray images in 100 s. CONCLUSIONS: We confirmed that intra-operative navigation using augmented reality provides an alternative way to perform distal locking in a safe and timely manner.

Multi-modal intra-operative navigation during distal locking of intramedullary nails.

The interlocking of intramedullary nails is a technically demanding procedure which involves a considerable amount of X-ray acquisitions; one study lists as many as 48 to successfully complete the procedure and fix screws into 4-6 mm distal holes of the nail. We propose to design an augmented radiolucent drill to assist surgeons in completing the distal locking procedure without any additional X-ray acquisitions. Using an augmented reality fluoroscope that coregisters optical and X-ray images, we exploit solely the optical images to detect the augmented radiolucent drill and estimate its tip position in real-time. Consequently, the surgeons will be able to maintain the down the beam positioning required to drill the screws into the nail holes successfully. To evaluate the accuracy of the proposed augmented drill, we perform a preclinical study involving six surgeons and ask them to perform distal locking on dry bone phantoms. Surgeons completed distal locking 98.3% of the time using only a single X-ray image with an average navigation time of 1.4 ± 0.9 min per hole.

Machine learning-based augmented reality for improved surgical scene understanding.

In orthopedic and trauma surgery, AR technology can support surgeons in the challenging task of understanding the spatial relationships between the anatomy, the implants and their tools. In this context, we propose a novel augmented visualization of the surgical scene that mixes intelligently the different sources of information provided by a mobile C-arm combined with a Kinect RGB-Depth sensor. Therefore, we introduce a learning-based paradigm that aims at (1) identifying the relevant objects or anatomy in both Kinect and X-ray data, and (2) creating an object-specific pixel-wise alpha map that permits relevance-based fusion of the video and the X-ray images within one single view. In 12 simulated surgeries, we show very promising results aiming at providing for surgeons a better surgical scene understanding as well as an improved depth perception.

Radiation-free drill guidance in interlocking of intramedullary nails.

Intramedullary nailing is a technically demanding procedure which involves an excessive amount of x-ray acquisitions; one study lists as many as 48 to successfully complete the procedure. In this work, a novel low cost radiation-free drilling guide is designed to assist surgeons in completing the distal locking procedure without any x-ray acquisitions. Using an augmented reality fluoroscope that coregisters optical and x-ray images, we exploit solely the optical images to detect the drilling guide in order to estimate the tip position in real-time in x-ray. We tested over 200 random drill guide poses showing a mean tip-estimation error of 1.72 +/- 0.7mm which is significantly robust and accurate for the interlocking. In a preclinical study on dry bone phantom, three expert surgeons successfully completed the interlocking 56 out of 60 trials with no x-ray acquisition for guidance and an average time of 2 min.