An automated workflow based on hip shape improves personalized risk prediction for hip osteoarthritis in the CHECK study.

OBJECTIVE: To design an automated workflow for hip radiographs focused on joint shape and tests its prognostic value for future hip osteoarthritis. DESIGN: We used baseline and 8-year follow-up data from 1,002 participants of the CHECK-study. The primary outcome was definite radiographic hip osteoarthritis (rHOA) (Kellgren-Lawrence grade ≥2 or joint replacement) at 8-year follow-up. We designed a method to automatically segment the hip joint from radiographs. Subsequently, we applied machine learning algorithms (elastic net with automated parameter optimization) to provide the Shape-Score, a single value describing the risk for future rHOA based solely on joint shape. We built and internally validated prediction models using baseline demographics, physical examination, and radiologists scores and tested the added prognostic value of the Shape-Score using Area-Under-the-Curve (AUC). Missing data was imputed by multiple imputation by chained equations. Only hips with pain in the corresponding leg were included. RESULTS: 84% were female, mean age was 56 (±5.1) years, mean BMI 26.3 (±4.2). Of 1,044 hips with pain at baseline and complete follow-up, 143 showed radiographic osteoarthritis and 42 were replaced. 91.5% of the hips had follow-up data available. The Shape-Score was a significant predictor of rHOA (odds ratio per decimal increase 5.21, 95%-CI (3.74-7.24)). The prediction model using demographics, physical examination, and radiologists scores demonstrated an AUC of 0.795, 95%-CI (0.757-0.834). After addition of the Shape-Score the AUC rose to 0.864, 95%-CI (0.833-0.895). CONCLUSIONS: Our Shape-Score, automatically derived from radiographs using a novel machine learning workflow, may strongly improve risk prediction in hip osteoarthritis.

Measuring vision using innate behaviours in mice with intact and impaired retina function.

Measuring vision in rodents is a critical step for understanding vision, improving models of human disease, and developing therapies. Established behavioural tests for perceptual vision, such as the visual water task, rely on learning. The learning process, while effective for sighted animals, can be laborious and stressful in animals with impaired vision, requiring long periods of training. Current tests that that do not require training are based on sub-conscious, reflex responses (e.g. optokinetic nystagmus) that don't require involvement of visual cortex and higher order thalamic nuclei. A potential alternative for measuring vision relies on using visually guided innate defensive responses, such as escape or freeze, that involve cortical and thalamic circuits. In this study we address this possibility in mice with intact and degenerate retinas. We first develop automatic methods to detect behavioural responses based on high dimensional tracking and changepoint detection of behavioural time series. Using those methods, we show that visually guided innate responses can be elicited using parametisable stimuli, and applied to describing the limits of visual acuity in healthy animals and discriminating degrees of visual dysfunction in mouse models of retinal degeneration.

Synovial volume vs synovial measurements from dynamic contrast enhanced MRI as measures of response in osteoarthritis.

OBJECTIVE: Synovium is increasingly a target of osteoarthritis (OA) treatment, yet its optimal measurement is unclear. Using dynamic contrast enhanced (DCE) MRI in knee OA patients before and after intraarticular steroid injection, we compared the responsiveness of static synovial volume measures to measures of dynamic changes in synovial enhancement, changes that are strongly related to synovial vascularity. METHODS: Ninety three patients underwent DCE-MRI before and 1-2 weeks after intra-articular injection of 80 mg methylprednisolone. Synovium was segmented and volume, relative enhancement rate (RER), maximum relative enhancement (REmax), late relative enhancement (RElate) and pharmacokinetic parameters (K(trans), ve) were calculated. KOOS (​knee injury and osteoarthritis outcome score) pain score was recorded before and after injection. Standardized change scores were calculated for each parameter. Linear regression and Pearson's correlations were used to investigate the relationship between change in MRI parameters and change in pain. RESULTS: The change in standardized score for the measures of synovial enhancement, RElate and REmax were -0.58 (95% CI -0.79 to -0.37) and -0.62 (95% CI -0.83 to -0.41) respectively, whereas the score for synovial volume was -0.30 (-0.52 to -0.09). Further, change in knee pain correlated more strongly with changes in enhancement (for both REmax and RElate, r = -0.27 (95% CI -0.45 to -0.07)) than with changes in synovial volume -0.15 (-0.35 to 0.05). CONCLUSION: This study suggests DCE-MRI derived measures of synovial enhancement may be more sensitive to the response to treatment and more strongly associated with changes in pain than synovial volume and may be better outcomes for assessment of structural effects of treatment in OA.

Increasing shape modelling accuracy by adjusting for subject positioning: an application to the analysis of radiographic proximal femur symmetry using data from the Osteoarthritis Initiative.

In total hip arthroplasty, the shape of the contra-lateral femur frequently serves as a template for preoperative planning. Previous research on contra-lateral femoral symmetry has been based on conventional hip geometric measurements (which reduce shape to a series of linear measurements) and did not take the effect of subject positioning on radiographic femur shape into account. The aim of this study was to analyse proximal femur symmetry based on statistical shape models (SSMs) which quantify global femoral shape while also adjusting for differences in subject positioning during image acquisition. We applied our recently developed fully automatic shape model matching (FASMM) system to automatically segment the proximal femur from AP pelvic radiographs to generate SSMs of the proximal femurs of 1258 Caucasian females (mean age: 61.3 SD=9.0). We used a combined SSM (capturing the left and right femurs) to identify and adjust for shape variation attributable to subject positioning as well as a single SSM (including all femurs as left femurs) to analyse proximal femur symmetry. We also calculated conventional hip geometric measurements (head diameter, neck width, shaft width and neck-shaft angle) using the output of the FASMM system. The combined SSM revealed two modes that were clearly attributable to subject positioning. The average difference (mean point-to-curve distance) between left and right femur shape was 1.0mm before and 0.8mm after adjusting for these two modes. The automatic calculation of conventional hip geometric measurements after adjustment gave an average absolute percent asymmetry of within 3.1% and an average absolute difference of within 1.1mm or 2.9° for all measurements. We conclude that (i) for Caucasian females the global shape of the right and left proximal femurs is symmetric without isolated locations of asymmetry; (ii) a combined left-right SSM can be used to adjust for radiographic shape variation due to subject positioning; and (iii) adjusting for subject positioning increases the accuracy of predicting the shape of the contra-lateral hip.

Development of a fully automatic shape model matching (FASMM) system to derive statistical shape models from radiographs: application to the accurate capture and global representation of proximal femur shape.

OBJECTIVE: To evaluate the accuracy and sensitivity of a fully automatic shape model matching (FASMM) system to derive statistical shape models (SSMs) of the proximal femur from non-standardised anteroposterior (AP) pelvic radiographs. DESIGN: AP pelvic radiographs obtained with informed consent and appropriate ethical approval were available for 1105 subjects with unilateral hip osteoarthritis (OA) who had been recruited previously for The arcOGEN Study. The FASMM system was applied to capture the shape of the unaffected (i.e., without signs of radiographic OA) proximal femur from these radiographs. The accuracy and sensitivity of the FASMM system in calculating geometric measurements of the proximal femur and in shape representation were evaluated relative to validated manual methods. RESULTS: De novo application of the FASMM system had a mean point-to-curve error of less than 0.9 mm in 99% of images (n = 266). Geometric measurements generated by the FASMM system were as accurate as those obtained manually. The analysis of the SSMs generated by the FASMM system for male and female subject groups identified more significant differences (in five of 17 SSM modes after Bonferroni adjustment) in their global proximal femur shape than those obtained from the analysis of conventional geometric measurements. Multivariate gender-classification accuracy was higher when using SSM mode values (76.3%) than when using conventional hip geometric measurements (71.8%). CONCLUSIONS: The FASMM system rapidly and accurately generates a global SSM of the proximal femur from radiographs of varying quality and resolution. This system will facilitate complex morphometric analysis of global shape variation across large datasets. The FASMM system could be adapted to generate SSMs from the radiographs of other skeletal structures such as the hand, knee or pelvis.

Fully automatic segmentation of the proximal femur using random forest regression voting.

Extraction of bone contours from radiographs plays an important role in disease diagnosis, preoperative planning, and treatment analysis. We present a fully automatic method to accurately segment the proximal femur in anteroposterior pelvic radiographs. A number of candidate positions are produced by a global search with a detector. Each is then refined using a statistical shape model together with local detectors for each model point. Both global and local models use Random Forest regression to vote for the optimal positions, leading to robust and accurate results. The performance of the system is evaluated using a set of 839 images of mixed quality. We show that the local search significantly outperforms a range of alternative matching techniques, and that the fully automated system is able to achieve a mean point-to-curve error of less than 0.9 mm for 99% of all 839 images. To the best of our knowledge, this is the most accurate automatic method for segmenting the proximal femur in radiographs yet reported.

Semi-automatic determination of detailed vertebral shape from lumbar radiographs using active appearance models.

SUMMARY: The vertebral endplates in lumbar radiographs were located by a semi-automatic annotation method using statistical shape models. INTRODUCTION: Vertebral fractures are common osteoporotic fractures, but current quantitative detection methods (morphometry) lack specificity. We have previously developed more specific quantitative classifiers of vertebral fracture using shape and appearance models. This method has only been applied to DXA vertebral fracture assessment (VFA) images and not to spinal radiographs. The classifiers require a detailed annotation of the outline of the vertebral endplate, so we investigated the application of similar semi-automated annotation methods to lumbar radiographs as the initial step in the generalisation of the statistical classifiers used in VFA images. METHODS: The vertebral body outlines in a training set of 670 lumbar radiographs were manually annotated by expert radiologists. This training set was used to build statistical models of vertebral shape and appearance using triplets of vertebrae. In order to segment vertebrae, the models were refitted using a sequence of active appearance models of vertebral triplets, using a miss-40-out train/test cross-validation experiment. The accuracy was evaluated against the manual annotation and analysed by fracture grade. RESULTS: Good accuracy was obtained for normal vertebrae (0.82 mm) and fracture grades 1 and 2 (1.19 mm), but the localisation accuracy deteriorated for grade 3 fractures to 2.12 mm. CONCLUSION: Vertebral body shape annotation can be substantially automated on lumbar radiographs. However, an occasional manual correction may be required, typically with either severe fractures, or when there is a high degree of projectional tilting or scoliosis. The located detailed shapes may enable the development of more powerful quantitative classifiers of osteoporotic vertebral fracture.

Detection of vertebral fractures in DXA VFA images using statistical models of appearance and a semi-automatic segmentation.

SUMMARY: Morphometric methods of vertebral fracture diagnosis lack specificity. We used detailed shape and image texture model parameters to improve the specificity of quantitative fracture identification. Two radiologists visually classified all vertebrae for system training and evaluation. The vertebral endplates were located by a semi-automatic segmentation method to obtain classifier inputs. INTRODUCTION: Vertebral fractures are common osteoporotic fractures, but current quantitative detection methods (morphometry) lack specificity. We used detailed shape and texture information to develop more specific quantitative classifiers of vertebral fracture to improve the objectivity of vertebral fracture diagnosis. These classifiers require a detailed segmentation of the vertebral endplate, and so we investigated the use of semi-automated segmentation methods as part of the diagnosis. METHODS: The vertebrae in a training set of 360 dual energy X-ray absorptiometry images were manually segmented. The shape and image texture of vertebrae were statistically modelled using Appearance Models. The vertebrae were given a gold standard classification by two radiologists. Linear discriminant classifiers to detect fractures were trained on the vertebral appearance model parameters. Classifier performance was evaluated by cross-validation for manual and semi-automatic segmentations, the latter derived using Active Appearance Models (AAM). Results were compared with a morphometric algorithm using the signs test. RESULTS: With manual segmentation, the false positive rates (FPR) at 95% sensitivity were: 5% (appearance) and 18% (morphometry). With semi-automatic segmentations the sensitivities at 5% FPR were: 88% (appearance) and 79% (morphometry). CONCLUSION: Specificity and sensitivity are improved by using an appearance-based classifier compared to standard height ratio morphometry. An overall sensitivity loss of 7% occurs (at 95% specificity) when using a semi-automatic (AAM) segmentation compared to expert annotation, due to segmentation error. However, the classifier sensitivity is still adequate for a computer-assisted diagnosis system for vertebral fracture, especially if used in a triage approach.

3D brain segmentation using active appearance models and local regressors.

We describe an efficient and accurate method for segmenting sets of subcortical structures in 3D MR images of the brain. We first find the approximate position of all the structures using a global Active Appearance Model (AAM). We then refine the shape and position of each structure using a set of individual AAMs trained for each. Finally we produce a detailed segmentation by computing the probability that each voxel belongs to the structure, using regression functions trained for each individual voxel. The models are trained using a large set of labelled images, using a novel variant of 'groupwise' registration to obtain the necessary image correspondences. We evaluate the method on a large dataset, and demonstrate that it achieves results comparable with some of the best published.

Comparison and evaluation of segmentation techniques for subcortical structures in brain MRI.

The automation of segmentation of medical images is an active research area. However, there has been criticism of the standard of evaluation of methods. We have comprehensively evaluated four novel methods of automatically segmenting subcortical structures using volumetric, spatial overlap and distance-based measures. Two of the methods are atlas-based - classifier fusion and labelling (CFL) and expectation-maximisation segmentation using a dynamic brain atlas (EMS), and two model-based - profile active appearance models (PAM) and Bayesian appearance models (BAM). Each method was applied to the segmentation of 18 subcortical structures in 270 subjects from a diverse pool varying in age, disease, sex and image acquisition parameters. Our results showed that all four methods perform on par with recently published methods. CFL performed significantly better than the other three methods according to all three classes of metrics.

Comparing the similarity of statistical shape models using the Bhattacharya metric.

A variety of different methods of finding correspondences across sets of images to build statistical shape models have been proposed, each of which is likely to result in a different model. When dealing with large datasets (particularly in 3D), it is difficult to evaluate the quality of the resulting models. However, if the different methods are successfully modelling the true underlying shape variation, the resulting models should be similar. If two different techniques lead to similar models, it suggests that they are indeed approximating the true shape change. In this paper we explore a method of comparing statistical shape models by evaluating the Bhattacharya overlap between their implied shape distributions. We apply the technique to investigate the similarity of three models of the same 3D dataset constructed using different methods.

Vertebral shape: automatic measurement with dynamically sequenced active appearance models.

The shape and appearance of vertebrae on lateral dual x-ray absorptiometry (DXA) scans were statistically modelled. The spine was modelled by a sequence of overlapping triplets of vertebrae, using Active Appearance Models (AAMs). To automate vertebral morphometry, the sequence of trained models was matched to previously unseen scans. The dataset includes a significant number of pathologies. A new dynamic ordering algorithm was assessed for the model fitting sequence, using the best quality of fit achieved by multiple sub-model candidates. The accuracy of the search was improved by dynamically imposing the best quality candidate first. The results confirm the feasibility of substantially automating vertebral morphometry measurements even with fractures or noisy images.

Anatomical statistical models and their role in feature extraction.

A detailed model of the shape of anatomical structures can significantly improve the ability to segment such structures from medical images. Statistical models representing the variation of shape and appearance can be constructed from suitably annotated training sets. Such models can be used to synthesize images of anatomy, and to search new images to accurately locate the structures of interest, even in the presence of noise and clutter. In this paper we summarize recent work on constructing and using such models, and demonstrate their application to several domains.

Improving appearance model matching using local image structure.

We show how non-linear representations of local image structure can be used to improve the performance of model matching algorithms in medical image analysis tasks. Rather than represent the image structure using intensity values or gradients, we use measures that indicate the reliability of a set of local image feature detector outputs. These features are image edges, corners, and gradients. Feature detector outputs in flat, noisy regions tend to be ignored whereas those near strong structure are favoured. We demonstrate that combinations of these features give more accurate and reliable matching between models and new images than modelling image intensity alone. We also show that the approach is robust to non-linear changes in contrast, such as those found in multi-modal imaging.

Model-based interpretation of complex and variable images.

The ultimate goal of machine vision is image understanding-the ability not only to recover image structure but also to know what it represents. By definition, this involves the use of models which describe and label the expected structure of the world. Over the past decade, model-based vision has been applied successfully to images of man-made objects. It has proved much more difficult to develop model-based approaches to the interpretation of images of complex and variable structures such as faces or the internal organs of the human body (as visualized in medical images). In such cases it has been problematic even to recover image structure reliably, without a model to organize the often noisy and incomplete image evidence. The key problem is that of variability. To be useful, a model needs to be specific-that is, to be capable of representing only 'legal' examples of the modelled object(s). It has proved difficult to achieve this whilst allowing for natural variability. Recent developments have overcome this problem; it has been shown that specific patterns of variability in shape and grey-level appearance can be captured by statistical models that can be used directly in image interpretation. The details of the approach are outlined and practical examples from medical image interpretation and face recognition are used to illustrate how previously intractable problems can now be tackled successfully. It is also interesting to ask whether these results provide any possible insights into natural vision; for example, we show that the apparent changes in shape which result from viewing three-dimensional objects from different viewpoints can be modelled quite well in two dimensions; this may lend some support to the 'characteristic views' model of natural vision.

Magnetic resonance image synthesis using a flexible model.

Image synthesis methods are based on the hypothesis that a magnetic resonance (MR) image with optimized contrast can be reproduced by synthesis from three calculated basic images of T1, T2 and spin density. This method, however, is limited by noise due to uncertainties in the initial measurements. The principal component analysis (PCA) method is based on an information theory approach that decomposes MR images into a small set of characteristic feature images. PCA images, or eigenimages, show morphology by condensing the structural information from the source images. Eigenimages have also been shown to improve contrast-to-noise ratio (CNR) compared with source images. In this study we have developed a method of synthesizing MR images using a flexible model, comprising a set of eigenimages derived from PCA. A matching process has been carried out to find the best fit between the model and a synthetic image calculated from the Bloch equations. The method has been applied to MR images obtained from a group of patients with intracranial lesions. The images derived from the flexible model show increased lesion conspicuity, reduced artefact and comparable CNR to the directly acquired images while maintaining the MR characteristic information for diagnosis.

Medical image interpretation: a generic approach using deformable templates.

We describe a generic approach to image interpretation, based on combining a general method of building flexible template models with genetic algorithm (GA) search. The method can be applied to a given image interpretation problem simply by training a statistical shape model, using a set of examples of the image structure to be located. A local optimization technique has been incorporated into the GA search and shown to improve the speed of convergence and optimality of solution. We present results from three medical applications, demonstrating that the new method offers significant improvements when compared with previously reported approaches to flexible template matching, particularly the ability to deal with different domains of application using a standard method and the possibility of employing complex multipart models. We also describe how the method can be simply extended to track structures in image sequences and segment three dimensional objects in volume images.