Feature fusion of Raman chemical imaging and digital histopathology using machine learning for prostate cancer detection.

The diagnosis of prostate cancer is challenging due to the heterogeneity of its presentations, leading to the over diagnosis and treatment of non-clinically important disease. Accurate diagnosis can directly benefit a patient's quality of life and prognosis. Towards addressing this issue, we present a learning model for the automatic identification of prostate cancer. While many prostate cancer studies have adopted Raman spectroscopy approaches, none have utilised the combination of Raman Chemical Imaging (RCI) and other imaging modalities. This study uses multimodal images formed from stained Digital Histopathology (DP) and unstained RCI. The approach was developed and tested on a set of 178 clinical samples from 32 patients, containing a range of non-cancerous, Gleason grade 3 (G3) and grade 4 (G4) tissue microarray samples. For each histological sample, there is a pathologist labelled DP-RCI image pair. The hypothesis tested was whether multimodal image models can outperform single modality baseline models in terms of diagnostic accuracy. Binary non-cancer/cancer models and the more challenging G3/G4 differentiation were investigated. Regarding G3/G4 classification, the multimodal approach achieved a sensitivity of 73.8% and specificity of 88.1% while the baseline DP model showed a sensitivity and specificity of 54.1% and 84.7% respectively. The multimodal approach demonstrated a statistically significant 12.7% AUC advantage over the baseline with a value of 85.8% compared to 73.1%, also outperforming models based solely on RCI and mean and median Raman spectra. Feature fusion of DP and RCI does not improve the more trivial task of tumour identification but does deliver an observed advantage in G3/G4 discrimination. Building on these promising findings, future work could include the acquisition of larger datasets for enhanced model generalization.

Robust colour calibration of an imaging system using a colour space transform and advanced regression modelling.

A new algorithm for the conversion of device dependent RGB colour data into device independent L*a*b* colour data without introducing noticeable error has been developed. By combining a linear colour space transform and advanced multiple regression methodologies it was possible to predict L*a*b* colour data with less than 2.2 colour units of error (CIE 1976). By transforming the red, green and blue colour components into new variables that better reflect the structure of the L*a*b* colour space, a low colour calibration error was immediately achieved (ΔE(CAL) = 14.1). Application of a range of regression models on the data further reduced the colour calibration error substantially (multilinear regression ΔE(CAL) = 5.4; response surface ΔE(CAL) = 2.9; PLSR ΔE(CAL) = 2.6; LASSO regression ΔE(CAL) = 2.1). Only the PLSR models deteriorated substantially under cross validation. The algorithm is adaptable and can be easily recalibrated to any working computer vision system. The algorithm was tested on a typical working laboratory computer vision system and delivered only a very marginal loss of colour information ΔE(CAL) = 2.35. Colour features derived on this system were able to safely discriminate between three classes of ham with 100% correct classification whereas colour features measured on a conventional colourimeter were not.

Correlation of consumer assessment of longissimus dorsi beef palatability with image colour, marbling and surface texture features.

A new study was conducted to apply computer vision methods successfully developed using trained sensory panel palatability data to new samples with consumer panel palatability data. The computer vision methodology utilized the traditional approach of using beef muscle colour, marbling and surface texture as palatability indicators. These features were linked to corresponding consumer panel palatability data with the traditional approach of partial least squares regression (PLSR). Best subsets were selected by genetic algorithms. Results indicate that accurate modelling of likeability with regression models was possible (r(2)=0.86). Modelling of other important palatability attributes proved encouraging (tenderness r(2)=0.76, juiciness r(2)=0.69, flavour r(2)=0.78). Therefore, the current study provides a basis for further expanding computer vision methodology to correlate with consumer panel palatability data.

Identification of important image features for pork and turkey ham classification using colour and wavelet texture features and genetic selection.

A method to discriminate between various grades of pork and turkey ham was developed using colour and wavelet texture features. Image analysis methods originally developed for predicting the palatability of beef were applied to rapidly identify the ham grade. With high quality digital images of 50-94 slices per ham it was possible to identify the greyscale that best expressed the differences between the various ham grades. The best 10 discriminating image features were then found with a genetic algorithm. Using the best 10 image features, simple linear discriminant analysis models produced 100% correct classifications for both pork and turkey on both calibration and validation sets.

Comparison of various wavelet texture features to predict beef palatability.

The wavelet transform can be used to characterise the surface texture of beef images in a more efficient manner than classical algorithms such as co-occurrence and run lengths. Features extracted from wavelet decompositions have been used to develop predictive models of important palatability attributes. A variety of common wavelet transforms were considered (biorthogonal, reverse biorthogonal, discrete Meyer, Daubechie, symmetric modified Daubechie and Coifman modified Daubechie) to search for the most useful texture features. A classic run length and co-occurrence algorithm was used for comparison. Using the same data analysis methods for each wavelet type, predictive models of beef acceptability, tenderness, juiciness, flavour and hardness were developed. Genetic algorithms succeeded in finding more accurate models than stepwise and manual elimination except for hardness. An accurate model of flavour (r(2)=0.84) was computed. A good model of overall acceptability (r(2)=0.79) was computed that fell just short of an important benchmark of accuracy. An encouraging model of juiciness (r(2)=0.71) was computed showing that with additional palatability information juiciness might be accurately modelled. Tenderness proved difficult to model with only the classic model satisfying stability criteria and a poorer result (r(2)=0.64) meaning substantial additional palatability information is required for accurate modelling. Hardness was particularly difficult to model. The biorthogonal wavelet produced the best model for three palatability measurements but the symmetric modified Daubechie wavelet produced the best model of overall acceptability and thus must be viewed as the most useful wavelet type.

Comparison of the predictive power of beef surface wavelet texture features at high and low magnification.

Beef longissimus dorsi surface texture is an indicator used in predicting beef palatability by expert graders. Computer vision systems have previously used imaging at normal view to develop surface texture features with some success. Good models of beef overall acceptability using imaging at high magnification have been recently developed. As a comparison the same surface texture features were computed from the corresponding images at normal view and used to model overall acceptability. Both sets of texture features were also combined with muscle colour and marbling features and used to model overall acceptability. Models using texture features alone were more successful at normal modality. However colour and marbling features combined much better with texture features at high modality to yield the most accurate model of overall acceptability (r(2)=0.93). Accurate Partial Least Squares Regression (PLSR) models were computed at both modalities with and without inclusion of colour and marbling features. Addition of squared terms to the models failed to improve accuracy.

Automatic segmentation of beef longissimus dorsi muscle and marbling by an adaptable algorithm.

An algorithm for automatic segmentation of beef longissimus dorsi (LD) muscle and marbling has been developed. The algorithm used simple thresholding to remove the background and then used clustering and thresholding with contrast enhancement via a customised greyscale to remove marbling. It was possible to attain lean muscle free of obvious marbling or background pixels where specular reflection could be effectively mitigated. Features of the automatically derived LD muscle and marbling images were compared to corresponding features of LD muscle and marbling images derived with a segmentation method requiring manual completion. Very strong correlations (up to r=1) were found between the colour features of both sets of LD muscle images. Strong correlations (up to r=0.96) were found between the features of both sets of marbling images. The automatic segmentation method has shown its good ability to approximate colour and marbling features. The algorithm has adaptable parameters and can be retailored to suit different image acquisition environments.

Development of a hybrid image processing algorithm for automatic evaluation of intramuscular fat content in beef M. longissimus dorsi.

An automatic method for estimating the content of intramuscular fat (IMF) in beef M. longissimus dorsi (LD) was developed using a sequence of image processing algorithm. To extract IMF particles within the LD muscle from structural features of intermuscular fat surrounding the muscle, three steps of image processing algorithm were developed, i.e. bilateral filter for noise removal, kernel fuzzy c-means clustering (KFCM) for segmentation, and vector confidence connected and flood fill for IMF extraction. The technique of bilateral filtering was firstly applied to reduce the noise and enhance the contrast of the beef image. KFCM was then used to segment the filtered beef image into lean, fat, and background. The IMF was finally extracted from the original beef image by using the techniques of vector confidence connected and flood filling. The performance of the algorithm developed was verified by correlation analysis between the IMF characteristics and the percentage of chemically extractable IMF content (P<0.05). Five IMF features are very significantly correlated with the fat content (P<0.001), including count densities of middle (CDMiddle) and large (CDLarge) fat particles, area densities of middle and large fat particles, and total fat area per unit LD area. The highest coefficient is 0.852 for CDLarge.

Prediction of beef eating quality from colour, marbling and wavelet texture features.

Beef longissimus dorsi colour, marbling fat and surface texture are long established properties that are used in some countries by expert graders to classify beef carcasses, with subjective and inconsistent decision. As a computer vision system can deliver objective and consistent decisions rapidly and is capable of handling a greater variety of image features, attempts have been made to develop computerised predictions of eating quality based on these and other properties but have failed to adequately model the variation in eating quality. Therefore, in this study, examination of the ribeye at high magnification and consideration of a broad range of colour and marbling fat features was used to attempt to provide better information on beef eating quality. Wavelets were used to describe the image texture of the beef surface at high magnification rather than classical methods such as run lengths, difference histograms and co-occurrence matrices. Sensory panel and Instron analyses were performed on duplicate steaks to measure the quality of the beef. Using the classical statistical method of partial least squares regression (PLSR) it was possible to model a very high proportion of the variation in eating quality (r(2)=0.88 for sensory overall acceptability and r(2)=0.85 for 7-day WBS). Addition of non-linear texture terms to the models gave some improvements.