Multi-Layer Scene Representation from Composed Focal Stacks.

Multi-layer images are a powerful scene representation for high-performance rendering in virtual/augmented reality (VR/AR). The major approach to generate such images is to use a deep neural network trained to encode colors and alpha values of depth certainty on each layer using registered multi-view images. A typical network is aimed at using a limited number of nearest views. Therefore, local noises in input images from a user-navigated camera deteriorate the final rendering quality and interfere with coherency over view transitions. We propose to use a focal stack composed of multi-view inputs to diminish such noises. We also provide theoretical analysis for ideal focal stacks to generate multi-layer images. Our results demonstrate the advantages of using focal stacks in coherent rendering, memory footprint, and AR-supported data capturing. We also show three applications of imaging for VR.

Improving compliance with guidelines may lead to favorable clinical outcomes for patients with non-muscle-invasive bladder cancer: A retrospective multicenter study.

OBJECTIVES: Clinical guidelines recommend that patients with non-muscle-invasive bladder cancer (NMIBC) should be treated with appropriate adjuvant therapy. However, compliance with guideline recommendations is insufficient, and this may lead to unfavorable outcomes. We aimed to investigate the level of adherence to guideline recommendations in patients with NMIBC and evaluate the outcomes of those who did and did not receive guideline-recommended therapies. METHODS: We performed a retrospective analysis of patients with histologically diagnosed NMIBC. The percentage of patients with intermediate- and high-risk tumors who received adjuvant intravesical therapy or second transurethral resection (TUR) was calculated. Recurrence-free survival was assessed in patients who did and did not receive the therapies. We conducted a propensity score-matched analysis to compare outcomes between patients with intermediate-risk and T1 NMIBC who did and did not undergo guideline-recommended therapies. RESULTS: Overall, 1204 patients from the Tohoku Urological Evidence-Based Medicine Study Group and Kyoto University Hospital were included. Of patients with intermediate- and high-risk tumors, 91.0% and 74.0% did not receive maintenance bacillus Calmette-Guérin (BCG), respectively. In both groups, significantly better recurrence-free survival was found for patients treated with maintenance BCG. Among patients with T1 NMIBC, only 16.7% underwent guideline-recommended therapies, that is, a second TUR and maintenance BCG. Significantly greater recurrence-free survival was observed in patients who received guideline-recommended therapies compared with propensity-matched patients who did not. CONCLUSIONS: Guideline-recommended therapies may contribute to improvements in outcomes for patients with NMIBC, suggesting that improvements in adherence to clinical guidelines may lead to favorable outcomes.

Screening for degenerative cervical myelopathy with the 10-second grip-and-release test using a smartphone and machine learning: A pilot study.

Objective: Early detection and intervention are essential for the mitigation of degenerative cervical myelopathy (DCM). However, although several screening methods exist, they are difficult to understand for community-dwelling people, and the equipment required to set up the test environment is expensive. This study investigated the viability of a DCM-screening method based on the 10-second grip-and-release test using a machine learning algorithm and a smartphone equipped with a camera to facilitate a simple screening system. Methods: Twenty-two participants comprising a group of DCM patients and 17 comprising a control group participated in this study. A spine surgeon diagnosed the presence of DCM. Patients performing the 10-second grip-and-release test were filmed, and the videos were analyzed. The probability of the presence of DCM was estimated using a support vector machine algorithm, and sensitivity, specificity, and area under the curve (AUC) were calculated. Two assessments of the correlation between estimated scores were conducted. The first used a random forest regression model and the Japanese Orthopaedic Association scores for cervical myelopathy (C-JOA). The second assessment used a different model, random forest regression, and the Disabilities of the Arm, Shoulder, and Hand (DASH) questionnaire. Results: The final classification model had a sensitivity of 90.9%, specificity of 88.2%, and AUC of 0.93. The correlations between each estimated score and the C-JOA and DASH scores were 0.79 and 0.67, respectively. Conclusions: The proposed model could be a helpful screening tool for DCM as it showed excellent performance and high usability for community-dwelling people and non-spine surgeons.

Future Prediction of Shuttlecock Trajectory in Badminton Using Player's Information.

Video analysis has become an essential aspect of net sports, such as badminton. Accurately predicting the future trajectory of balls and shuttlecocks can significantly benefit players by enhancing their performance and enabling them to devise effective game strategies. This paper aims to analyze data to provide players with an advantage in the fast-paced rallies of badminton matches. The paper delves into the innovative task of predicting future shuttlecock trajectories in badminton match videos and presents a method that takes into account both the shuttlecock position and the positions and postures of the players. In the experiments, players were extracted from the match video, their postures were analyzed, and a time-series model was trained. The results indicate that the proposed method improved accuracy by 13% compared to methods that solely used shuttlecock position information as input, and by 8.4% compared to methods that employed both shuttlecock and player position information as input.

Real-world outcomes of patients with renal cell carcinoma, surgically treated at regional hospitals, based on a prospective long-term survey of the pre-robotic era.

PURPOSE: Renal cancer surgery is frequently performed in small regional hospitals in Japan. This study evaluated the outcomes of renal cancer surgery, comparing results from the pre-robotic surgery era with those obtained with robotic surgery. METHODS: This prospective cohort study was conducted on patients who underwent renal cancer surgery between 2008 and 2013 at 14 hospitals, comprising 13 regional hospitals and a university hospital, registered in the Tohoku Urological Evidence-Based Medicine Study Group. The patients' backgrounds; perioperative data; annual postoperative renal function; and prognostic surveys, performed over a median follow-up period of 10 years were obtained. RESULTS: In 930 surgical cases at the 14 registered hospitals, the 10-year recurrence-free survival rates of cT1a, cT1b, cT2, and cT3 were 0.9326, 0.8501, 0.5786, and 0.5101, respectively. Meanwhile, the 10-year overall survival rates were 0.9612, 0.8662, 0.7505, and 0.7209, respectively. Long-term observation in patients with cT1 showed that vessel involvement and high tumor grade were prognostic factors for recurrence. As a noteworthy fact, radical nephrectomy was performed in 53.3% of patients with cT1a at the regional hospitals. However, even in patients with preoperative chronic kidney disease stage 3, radical nephrectomy was not a prognostic factor of renal function. This indicates that compensatory mechanisms had been working for a long time in many patients who underwent radical nephrectomies without hypertension and preoperative proteinuria, which were predictors of end-stage renal disease. CONCLUSION: Based on a prospective long-term survey of the pre-robotic era, our results suggested no difference of the survival outcomes between the university hospital and regional hospitals. Our study provides baseline data to evaluate the outcomes of renal cancer robotic surgery, performed at regional hospitals.

Object Pose Estimation Using Edge Images Synthesized from Shape Information.

This paper presents a method for estimating the six Degrees of Freedom (6DoF) pose of texture-less objects from a monocular image by using edge information. The deep learning-based pose estimation method needs a large dataset containing pairs of an image and ground truth pose of objects. To alleviate the cost of collecting a dataset, we focus on the method using a dataset made by computer graphics (CG). This simulation-based method prepares a thousand images by rendering the computer-aided design (CAD) data of the object and trains a deep-learning model. As an inference stage, a monocular RGB image is entered into the model, and the object's pose is estimated. The representative simulation-based method, Pose Interpreter Networks, uses silhouette images as the input, thereby enabling common feature (contour) extraction from RGB and CG images. However, estimating rotation parameters is less accurate. To overcome this problem, we propose a method to use edge information extracted from the object's ridgelines for training the deep learning model. Since edge distribution changes largely according to the pose, the estimation of rotation parameters becomes more robust. Through an experiment with simulation data, we quantitatively proved the accuracy improvement compared to the previous method (error rate decreases at a certain condition are translation 22.9% and rotation: 43.4%). Moreover, through an experiment with physical data, we clarified the issues of this method and proposed an effective solution by fine-tuning (error rate decrease at a certain condition are translation 20.1% and rotation 57.7%).

Minimal residual membranous urethral length and membranous urethral length predict poor recovery from incontinence after robot-assisted radical prostatectomy and after open radical prostatectomy.

OBJECTIVES: To investigate how much minimal residual membranous urethral length (mRUL) and maximal urethral length (MUL) measured on MRI preoperatively affect postoperative urinary incontinence (PUI) and recovery in robot-assisted radical prostatectomy (RARP) and open radical prostatectomy (ORP). METHODS: The subjects were 190 and 110 patients undergoing RARP and ORP, respectively, in our institution. Patients underwent preoperative MRI for prostate cancer evaluation and completed the quality of life questionnaire of the Expanded Prostate Cancer Index Composite instrument before and 1, 3, 6, and 12 months after surgery. The parameters of mRUL and MUL were measured on MRI and analyzed along with other parameters including age, body mass index, and nerve sparing. RESULTS: The median mRUL and MUL were 7.81 and 14.27 mm in the RARP group and 7.15 and 13.57 mm in the ORP group, respectively. Recovery rates from PUI were similar in the two groups. Multivariate analyses showed that mRUL was a predictor of baseline continence, whereas shorter MUL was a predictor of poor recovery from PUI. Patients with both shorter mRUL and MUL had significantly worse recoveries from PUI after RARP and ORP than patients with longer mRUL and MUL. CONCLUSIONS: Minimal residual membranous urethral length contributes to urethral function as basal urinary continence, whereas MUL represents the potential of recovery from PUI in RARP and ORP. The MUL measured by preoperative MRI can predict poor recovery from PUI after radical prostatectomy and combined evaluation of MUL and mRUL support to anticipate poor recovery of PUI.

Multi-Camera Multi-Person Tracking and Re-Identification in an Operating Room.

Multi-camera multi-person (MCMP) tracking and re-identification (ReID) are essential tasks in safety, pedestrian analysis, and so on; however, most research focuses on outdoor scenarios because they are much more complicated to deal with occlusions and misidentification in a crowded room with obstacles. Moreover, it is challenging to complete the two tasks in one framework. We present a trajectory-based method, integrating tracking and ReID tasks. First, the poses of all surgical members captured by each camera are detected frame-by-frame; then, the detected poses are exploited to track the trajectories of all members for each camera; finally, these trajectories of different cameras are clustered to re-identify the members in the operating room across all cameras. Compared to other MCMP tracking and ReID methods, the proposed one mainly exploits trajectories, taking texture features that are less distinguishable in the operating room scenario as auxiliary cues. We also integrate temporal information during ReID, which is more reliable than the state-of-the-art framework where ReID is conducted frame-by-frame. In addition, our framework requires no training before deployment in new scenarios. We also created an annotated MCMP dataset with actual operating room videos. Our experiments prove the effectiveness of the proposed trajectory-based ReID algorithm. The proposed framework achieves 85.44% accuracy in the ReID task, outperforming the state-of-the-art framework in our operating room dataset.

Accuracy and Speed Improvement of Event Camera Motion Estimation Using a Bird's-Eye View Transformation.

Event cameras are bio-inspired sensors that have a high dynamic range and temporal resolution. This property enables motion estimation from textures with repeating patterns, which is difficult to achieve with RGB cameras. Therefore, motion estimation of an event camera is expected to be applied to vehicle position estimation. An existing method, called contrast maximization, is one of the methods that can be used for event camera motion estimation by capturing road surfaces. However, contrast maximization tends to fall into a local solution when estimating three-dimensional motion, which makes correct estimation difficult. To solve this problem, we propose a method for motion estimation by optimizing contrast in the bird's-eye view space. Instead of performing three-dimensional motion estimation, we reduced the dimensionality to two-dimensional motion estimation by transforming the event data to a bird's-eye view using homography calculated from the event camera position. This transformation mitigates the problem of the loss function becoming non-convex, which occurs in conventional methods. As a quantitative experiment, we created event data by using a car simulator and evaluated our motion estimation method, showing an improvement in accuracy and speed. In addition, we conducted estimation from real event data and evaluated the results qualitatively, showing an improvement in accuracy.

Cervical Myelopathy Screening with Machine Learning Algorithm Focusing on Finger Motion Using Noncontact Sensor.

STUDY DESIGN: Cross-sectional study. OBJECTIVE: To develop a binary classification model for cervical myelopathy (CM) screening based on a machine learning algorithm using Leap Motion (Leap Motion, San Francisco, CA), a novel noncontact sensor device. SUMMARY OF BACKGROUND DATA: Progress of CM symptoms are gradual and cannot be easily identified by the patients themselves. Therefore, screening methods should be developed for patients of CM before deterioration of myelopathy. Although some studies have been conducted to objectively evaluate hand movements specific to myelopathy using cameras or wearable sensors, their methods are unsuitable for simple screening outside hospitals because of the difficulty in obtaining and installing their equipment and the long examination time. METHODS: In total, 50 and 28 participants in the CM and control groups were recruited, respectively. The diagnosis of CM was made by spine surgeons. We developed a desktop system using Leap Motion that recorded 35 parameters of fingertip movements while participants gripped and released their fingers as rapidly as possible. A support vector machine was used to develop the binary classification model, and a multiple linear regression analysis was performed to create regression models to estimate the total Japanese Orthopaedic Association (JOA) score and the JOA score of the motor function of the upper extremity (MU-JOA score). RESULTS: The binary classification model indexes were as follows: sensitivity, 84.0%; specificity, 60.7%; accuracy, 75.6%; area under the curve, 0.85. The Spearman rank correlation coefficient between the estimated score and the total JOA score was 0.44 and that between the estimated score and the MU-JOA score was 0.51. CONCLUSION: Our binary classification model using a machine learning algorithm and Leap Motion could classify CM with high sensitivity and would be useful for CM screening in daily life before consulting doctors and telemedicine.Level of Evidence: 3.

High-Dimensional Analysis of Finger Motion and Screening of Cervical Myelopathy With a Noncontact Sensor: Diagnostic Case-Control Study.

BACKGROUND: Cervical myelopathy (CM) causes several symptoms such as clumsiness of the hands and often requires surgery. Screening and early diagnosis of CM are important because some patients are unaware of their early symptoms and consult a surgeon only after their condition has become severe. The 10-second hand grip and release test is commonly used to check for the presence of CM. The test is simple but would be more useful for screening if it could objectively evaluate the changes in movement specific to CM. A previous study analyzed finger movements in the 10-second hand grip and release test using the Leap Motion, a noncontact sensor, and a system was developed that can diagnose CM with high sensitivity and specificity using machine learning. However, the previous study had limitations in that the system recorded few parameters and did not differentiate CM from other hand disorders. OBJECTIVE: This study aims to develop a system that can diagnose CM with higher sensitivity and specificity, and distinguish CM from carpal tunnel syndrome (CTS), a common hand disorder. We then validated the system with a modified Leap Motion that can record the joints of each finger. METHODS: In total, 31, 27, and 29 participants were recruited into the CM, CTS, and control groups, respectively. We developed a system using Leap Motion that recorded 229 parameters of finger movements while participants gripped and released their fingers as rapidly as possible. A support vector machine was used for machine learning to develop the binary classification model and calculated the sensitivity, specificity, and area under the curve (AUC). We developed two models, one to diagnose CM among the CM and control groups (CM/control model), and the other to diagnose CM among the CM and non-CM groups (CM/non-CM model). RESULTS: The CM/control model indexes were as follows: sensitivity 74.2%, specificity 89.7%, and AUC 0.82. The CM/non-CM model indexes were as follows: sensitivity 71%, specificity 72.87%, and AUC 0.74. CONCLUSIONS: We developed a screening system capable of diagnosing CM with higher sensitivity and specificity. This system can differentiate patients with CM from patients with CTS as well as healthy patients and has the potential to screen for CM in a variety of patients.

Hand Motion-Aware Surgical Tool Localization and Classification from an Egocentric Camera.

Detecting surgical tools is an essential task for the analysis and evaluation of surgical videos. However, in open surgery such as plastic surgery, it is difficult to detect them because there are surgical tools with similar shapes, such as scissors and needle holders. Unlike endoscopic surgery, the tips of the tools are often hidden in the operating field and are not captured clearly due to low camera resolution, whereas the movements of the tools and hands can be captured. As a result that the different uses of each tool require different hand movements, it is possible to use hand movement data to classify the two types of tools. We combined three modules for localization, selection, and classification, for the detection of the two tools. In the localization module, we employed the Faster R-CNN to detect surgical tools and target hands, and in the classification module, we extracted hand movement information by combining ResNet-18 and LSTM to classify two tools. We created a dataset in which seven different types of open surgery were recorded, and we provided the annotation of surgical tool detection. Our experiments show that our approach successfully detected the two different tools and outperformed the two baseline methods.

Deep learning in diabetic foot ulcers detection: A comprehensive evaluation.

There has been a substantial amount of research involving computer methods and technology for the detection and recognition of diabetic foot ulcers (DFUs), but there is a lack of systematic comparisons of state-of-the-art deep learning object detection frameworks applied to this problem. DFUC2020 provided participants with a comprehensive dataset consisting of 2,000 images for training and 2,000 images for testing. This paper summarizes the results of DFUC2020 by comparing the deep learning-based algorithms proposed by the winning teams: Faster R-CNN, three variants of Faster R-CNN and an ensemble method; YOLOv3; YOLOv5; EfficientDet; and a new Cascade Attention Network. For each deep learning method, we provide a detailed description of model architecture, parameter settings for training and additional stages including pre-processing, data augmentation and post-processing. We provide a comprehensive evaluation for each method. All the methods required a data augmentation stage to increase the number of images available for training and a post-processing stage to remove false positives. The best performance was obtained from Deformable Convolution, a variant of Faster R-CNN, with a mean average precision (mAP) of 0.6940 and an F1-Score of 0.7434. Finally, we demonstrate that the ensemble method based on different deep learning methods can enhance the F1-Score but not the mAP.

Development and User Evaluation of a Smartphone-Based System to Assess Range of Motion of Wrist Joint.

PURPOSE: Measuring range of motion (ROM) in the wrist joint is an essential part of hand and wrist functional evaluations, especially before and after surgery. However, accurate measurements require experience and time. To reduce patient and surgeon burdens related to ROM measurement, a smartphone-based system, which enables participants to measure the ROM of the wrist joint semiautomatically using self-taken pictures on a smartphone, was developed and evaluated in this study. METHODS: In the developed system, participants were asked to take a picture of their wrist by using the other hand to position the joint first into full flexion and then into full extension. The hand and arm regions were automatically extracted in the program, and the ROM was estimated after the area of the hand and forearm was cropped. To verify the accuracy of ROM measurements in this system, the proposed method was tested on 66 images of hands from 33 participants; measurements were compared with those taken by hand surgeons. A limit of agreement and an intraclass correlation coefficient (ICC) were used for evaluation. RESULTS: The smallest averages (95% limits of agreement) of flexion and extension were 11.32° (95% confidence interval [CI], 8.88° to 13.76°) and 11.01° (95% CI, 8.64° to 13.39°), respectively. The ICC (1,1) for 3 measurements taken by one assessor was 0.99 (95% CI, 0.986-0.992), and the ICC (2,1) for 2 measurements taken by both assessors was 0.97 (95% CI, 0.947-0.977). CONCLUSIONS: In this study, we developed a system to measure the semiautomatic ROM of the wrist joint using a smartphone image. Its accuracy was within a clinically usable error range that was comparable with that of a hand surgeon. CLINICAL RELEVANCE: This system can reduce the burden of ROM measurement for both patients and doctors.

InpaintFusion: Incremental RGB-D Inpainting for 3D Scenes.

State-of-the-art methods for diminished reality propagate pixel information from a keyframe to subsequent frames for real-time inpainting. However, these approaches produce artifacts, if the scene geometry is not sufficiently planar. In this article, we present InpaintFusion, a new real-time method that extends inpainting to non-planar scenes by considering both color and depth information in the inpainting process. We use an RGB-D sensor for simultaneous localization and mapping, in order to both track the camera and obtain a surfel map in addition to RGB images. We use the RGB-D information in a cost function for both the color and the geometric appearance to derive a global optimization for simultaneous inpainting of color and depth. The inpainted depth is merged in a global map by depth fusion. For the final rendering, we project the map model into image space, where we can use it for effects such as relighting and stereo rendering of otherwise hidden structures. We demonstrate the capabilities of our method by comparing it to inpainting results with methods using planar geometric proxies.

Resolving Position Ambiguity of IMU-Based Human Pose with a Single RGB Camera.

Human motion capture (MoCap) plays a key role in healthcare and human-robot collaboration. Some researchers have combined orientation measurements from inertial measurement units (IMUs) and positional inference from cameras to reconstruct the 3D human motion. Their works utilize multiple cameras or depth sensors to localize the human in three dimensions. Such multiple cameras are not always available in our daily life, but just a single camera attached in a smart IP devices has recently been popular. Therefore, we present a 3D pose estimation approach from IMUs and a single camera. In order to resolve the depth ambiguity of the single camera configuration and localize the global position of the subject, we present a constraint which optimizes the foot-ground contact points. The timing and 3D positions of the ground contact are calculated from the acceleration of IMUs on foot and geometric transformation of foot position detected on image, respectively. Since the results of pose estimation is greatly affected by the failure of the detection, we design the image-based constraints to handle the outliers of positional estimates. We evaluated the performance of our approach on public 3D human pose dataset. The experiments demonstrated that the proposed constraints contributed to improve the accuracy of pose estimation in single and multiple camera setting.

Overhead Multiview Camera System for Recording Open Surgery.

Supplemental Digital Content is available in the text.

[Two Case Reports of Penile Fracture : Consideration on Sexual Function].

Approximately 400 cases of penile fracture have been reported in Japan, but the sexual function before and after treatment has not been evaluated. Here, we show 2 surgical procedures dealing with penile fractureand examinethechange s in sexual functions using IIEF-5. Case1 was in a 51 year old malewho underwent emergency surgery for a penile fracture. The IIEF-5 score was 17 points before surgery and 8 points 2 months after surgery. At 5 months post-surgery, the patient complained of mild pain and penile curvature while erect, still the IIEF-5 score showed an improvement to 12 points. Case 2 was in a 60 year old male who underwent emergency surgery for penile fracture. The IIEF-5 score was 21 points before surgery and 8 points 2 months after surgery. Erection and ejaculation became possible 6 months after surgery, and the IIEF-5 score showed an improvement to 21 points. After surgery, the IIEF-5 score declined and sexual function also declined temporarily, though both gradually improved. From a sexual functioning standpoint, surgical treatment would be preferable.

Gait Measurement at Home Using A Single RGB Camera.

BACKGROUND: An individual's gait is a key factor for consideration in evaluating their overall health. Several medical studies have demonstrated the correlation between gait and incidence rate of diseases, mortality, and risk of fall. However, gait is only occasionally evaluated during medical visits, which may delay the detection of health problems. METHODS: In this paper, we propose a gait measurement system that is suitable for use at home. Our method requires only a single RGB camera, whereas other visionary sensor-based methods require depth sensors or multiple RGB cameras. In addition, the setup for the measurement is easy. What the user has to do is only putting a single camera in a room and choosing four location known points on the floor. Our method can measure step positions and step timings, and therefore, other important parameters such as stride length, step width, walking speed, and cadence may also be captured. The individual's gait is captured by the camera, and therefore the user is not required to wear any devices. RESULTS: In the experiment described herein, we demonstrate our method's accuracy by comparing it with the motion capture system. The results indicate that our method can measure walking speed with an error of 3.62 cm/s from the side view, and which is too small a change to be clinically meaningful.

Relationships between sunitinib plasma concentration and clinical outcomes in Japanese patients with metastatic renal cell carcinoma.

BACKGROUND: The aim was to investigate the relationships between total sunitinib plasma concentrations (sunitinib plus its active metabolite; N-desethyl sunitinib) and clinical outcomes in Japanese patients with metastatic renal cell carcinoma (mRCC). METHODS: Twenty patients with mRCC were enrolled following treatment with sunitinib. To assess safety, the total sunitinib concentration range up to discontinuation of treatment and dosage reduction associated with adverse events within 6 weeks from initiating administration were analyzed. The longest administered sunitinib dosage was defined as the maintenance dose, and the relationship between total sunitinib concentration at the maintenance dosage and sunitinib efficacy was investigated. RESULTS: Total sunitinib concentration was significantly higher in patients who discontinued treatment or had dosage reduction due to adverse events within 6 weeks after initiation of sunitinib than in patients who continued treatment with the initial dosage. The time to treatment failure, progression-free survival, and overall survival were better in patients with total sunitinib concentrations < 50 ng/mL than in those with concentrations ≥ 50 ng/mL. CONCLUSIONS: The present study demonstrated that the effective range of total sunitinib concentration in Japanese patients with mRCC was lower than 50-100 ng/mL which was previously reported. These results indicate that therapeutic drug monitoring could maintain the therapeutic effect of sunitinib while minimizing adverse events by personalizing sunitinib dosages for Japanese patients with mRCC.