Real-time biomechanical modelling of the liver using LightGBM model.

BACKGROUND: Accurate and real-time biomechanical modelling of the liver is a major challenge in computer-assisted surgery. Finite element method is often used to predict the deformation of organs for its high modelling accuracy. However, its high computation cost hinders its application in real time, such as virtual surgery simulations. METHOD: A liver model with biomechanical properties similar to real one is created using finite element method and a data set of the liver deformation with different forces (whose magnitude ranges from 0.1 to 0.5 N in omni-direction) acting on different surface points is generated. The mechanical behaviour of liver is simulated in real time by a tree-based LightGBM regression model trained with the generated data set. RESULTS: In comparison with the Random Forest and XGBoost, the LightGBM model achieves the best accuracy with 0.0774 mm, 0.0786 mm, 0.0801 mm in the mean absolute error (MAE) and 0.0591 mm, 0.0609 and 0.0622 mm in the root mean square error (RMSE) along x, y and z axis, respectively. In addition, it only takes 33 ms for the LightGBM model to estimate the deformation of the liver, which is much faster than finite element model (29.91 s). CONCLUSION: These results lay a foundation for the future development of real-time virtual surgery systems of simulating liver deformation during minimally invasive surgeries using our method.

Facial landmark-guided surface matching for image-to-patient registration with an RGB-D camera.

BACKGROUND: Fiducial marker-based image-to-patient registration is the most common way in image-guided neurosurgery, which is labour-intensive, time consuming, invasive and error prone. METHODS: We proposed a method of facial landmark-guided surface matching for image-to-patient registration using an RGB-D camera. Five facial landmarks are localised from preoperative magnetic resonance (MR) images using deep learning and RGB image using Adaboost with multi-scale block local binary patterns, respectively. The registration of two facial surface point clouds derived from MR images and RGB-D data is initialised by aligning these five landmarks and further refined by weighted iterative closest point algorithm. RESULTS: Phantom experiment results show the target registration error is less than 3 mm when the distance from the camera to the phantom is less than 1000 mm. The registration takes less than 10 s. CONCLUSIONS: The proposed method is comparable to the state-of-the-arts in terms of the accuracy yet more time-saving and non-invasive.

A non-contact interactive stereo display system for exploring human anatomy.

Stereoscopic display based on Virtual Reality (VR) can facilitate clinicians observing 3 D anatomical models with the depth cue which lets them understand the spatial relationship between different anatomical structures intuitively. However, there are few input devices available in the sterile field of the operating room for controlling 3 D anatomical models. This paper presents a cost-effective VR application for stereo display of 3 D anatomical models with non-contact interaction. The system is integrated with hand gesture interaction and voice interaction to achieve non-contact interaction. Hand gesture interaction is based on Leap Motion. Voice interaction is implemented based on Bing Speech for English language and Aitalk for Chinese language. A local database is designed to record the anatomical terminologies organized in a tree structure, and provided to the speech recognition engine for querying these uncommon words. Ten participants were asked to practice the proposed system and compare it with the common interactive manners. The results show that our system is more efficient than the common interactive manner and prove the feasibility and practicability of the proposed system used in the sterile field.