Hyperspectral Imaging in Brain Tumor Surgery-Evidence of Machine Learning-Based Performance.

BACKGROUND: Hyperspectral imaging (HSI) has the potential to enhance surgical tissue detection and diagnostics. Definite utilization of intraoperative HSI guidance demands validated machine learning and public datasets that currently do not exist. Moreover, current imaging conventions are dispersed, and evidence-based paradigms for neurosurgical HSI have not been declared. METHODS: We presented the rationale and a detailed clinical paradigm for establishing microneurosurgical HSI guidance. In addition, a systematic literature review was conducted to summarize the current indications and performance of neurosurgical HSI systems, with an emphasis on machine learning-based methods. RESULTS: The published data comprised a few case series or case reports aiming to classify tissues during glioma operations. For a multitissue classification problem, the highest overall accuracy of 80% was obtained using deep learning. Our HSI system was capable of intraoperative data acquisition and visualization with minimal disturbance to glioma surgery. CONCLUSIONS: In a limited number of publications, neurosurgical HSI has demonstrated unique capabilities in contrast to the established imaging techniques. Multidisciplinary work is required to establish communicable HSI standards and clinical impact. Our HSI paradigm endorses systematic intraoperative HSI data collection, which aims to facilitate the related standards, medical device regulations, and value-based medical imaging systems.

Detection improvement of gliomas in hyperspectral imaging of protoporphyrin IX fluorescence - in vitro comparison of visual identification and machine thresholds.

BACKGROUND: 5-aminolevulinic acid (5-ALA) - precursor of protoporphyrin IX (PpIX) - is utilized in fluorescence guided surgery (FGS) of high-grade gliomas. PpIX is used to identify traces of glioma during resection. Visual inspection of the fluorescence seems inaccurate in comparison to optic techniques such as hyperspectral imaging (HSI). AIM: To characterize the limits of PpIX fluorescence detection of (i) visual evaluation and (ii) HSI analysis and to (iii) develop a classification system for visible and non-visible PpIX fluorescence. METHODS: Samples with increasing concentrations (C) of PpIX and non-fluorescent controls were evaluated using a surgical microscope under blue light illumination. Similar samples were imaged with a HSI system tuned to PpIX fluorescence peak wavelength (635 nm) and control (RGB) channels. Samples' intensities were defined, leading to 96 analysed pixels after batching. RESULTS: Three expert neurosurgeons assessed the PpIX samples (n = 16) and controls (n = 8) with unanimous decisions (ICC = 0.704), resulting in 63% recognition rate, 48% sensitivity, 92% specificity, 92% positive predictive value (PPV) and 47% negative predictive value (NPV). HSI image analysis, comparing mean relative values, resulted in 96%, 100%, 86%, 94%, 100%, respectively. Minimum PpIX concentration detection for experts was 0.6-1.8 µmol/l and HSI's 0.03-0.15 µmol/l. CONCLUSIONS: PpIX concentrations of low-grade gliomas, and those reported on glioblastoma infiltration zones, are below experts' detection threshold. HSI analysis exceeds the performance of expert's visual inspection nearly by 20-fold. Hybrid FGS-HSI systems should be investigated in parallel to long-term outcomes. Described methods are applicable as a standard for calibration, testing and development of subvisual FGS techniques.

The Optimization of the Light-Source Spectrum Utilizing Neural Networks for Detecting Oral Lesions.

Any change in the light-source spectrum modifies the color information of an object. The spectral distribution of the light source can be optimized to enhance specific details of the obtained images; thus, using information-enhanced images is expected to improve the image recognition performance via machine vision. However, no studies have applied light spectrum optimization to reduce the training loss in modern machine vision using deep learning. Therefore, we propose a method for optimizing the light-source spectrum to reduce the training loss using neural networks. A two-class classification of one-vs-rest among the classes, including enamel as a healthy condition and dental lesions, was performed to validate the proposed method. The proposed convolutional neural network-based model, which accepts a 5 × 5 small patch image, was compared with an alternating optimization scheme using a linear-support vector machine that optimizes classification weights and lighting weights separately. Furthermore, it was compared with the proposed neural network-based algorithm, which inputs a pixel and consists of fully connected layers. The results of the five-fold cross-validation revealed that, compared to the previous method, the proposed method improved the F1-score and was superior to the models that were using the immutable standard illuminant D65.

Articular cartilage optical properties in the near-infrared (NIR) spectral range vary with depth and tissue integrity.

Optical properties of biological tissues in the NIR spectral range have demonstrated significant potential for in vivo diagnostic applications and are critical parameters for modelling light interaction in biological tissues. This study aims to investigate the optical properties of articular cartilage as a function of tissue depth and integrity. The results suggest consistent wavelength-dependent variation in optical properties between cartilage depth-wise zones, as well as between healthy and degenerated tissue. Also, statistically significant differences (p<0.05) in both optical properties were observed between the different cartilage depth-wise zones and as a result of tissue degeneration. When taken into account, the outcome of this study could enable accurate modelling of light interaction in cartilage matrix and could provide useful diagnostic information on cartilage integrity.

Tissue optical properties combined with machine learning enables estimation of articular cartilage composition and functional integrity.

Absorption and reduced scattering coefficients ( µ a , µ s ' ) of biological tissues have shown significant potential in biomedical applications. Thus, they are effective parameters for the characterization of tissue integrity and provide vital information on the health of biological tissues. This study investigates the potential of optical properties ( µ a , µ s ' ) for estimating articular cartilage composition and biomechanical properties using multivariate and machine learning techniques. The results suggest that µa could optimally estimate cartilage proteoglycan content in the superficial zone, in addition to its equilibrium modulus. While µ s ' could effectively estimate the proteoglycan content of the middle and deep zones in addition to the instantaneous and dynamic moduli of articular cartilage.

A Novel Approach to Using Spectral Imaging to Classify Dyes in Colored Fibers.

In the field of cultural heritage, applied dyes on textiles are studied to explore their great artistic and historic values. Dye analysis is essential and important to plan correct restoration, preservation and display strategy in museums and art galleries. However, most of the existing diagnostic technologies are destructive to the historical objects. In contrast to that, spectral reflectance imaging is potential as a non-destructive and spatially resolved technique. There have been hardly any studies in classification of dyes in textile fibers using spectral imaging. In this study, we show that spectral imaging with machine learning technique is capable in preliminary screening of dyes into the natural or synthetic class. At first, sparse logistic regression algorithm is applied on reflectance data of dyed fibers to determine some discriminating bands. Then support vector machine algorithm (SVM) is applied for classification considering the reflectance of the selected spectral bands. The results show nine selected bands in short wave infrared region (SWIR, 1000-2500 nm) classify dyes with 97.4% accuracy (kappa 0.94). Interestingly, the results show that fairly accurate dye classification can be achieved using the bands at 1480nm, 1640 nm, and 2330 nm. This indicates possibilities to build an inexpensive handheld screening device for field studies.

Optical implementation of partially negative filters using a spectrally tunable light source, and its application to contrast enhanced oral and dental imaging.

In optical imaging, optical filters can be used to enhance the visibility of features-of-interest and thus aid in visualization. Optical filter design based on hyperspectral imaging employs various statistical methods to find an optimal design. Some methods, like principal component analysis, produce vectors that can be interpreted as filters that have a partially negative transmission spectrum. These filters, however, are not directly implementable optically. Earlier implementations of partially negative filters have concentrated on spectral reconstruction. Here we show a novel method for implementing partially negative optical filters for contrast-enhancement purposes in imaging applications. We describe the method and its requirements, and show its feasibility with color chart and dental imaging examples. The results are promising: visual comparison of computational color chart render and optical measurement show matching images, and visual inspection of dental images show increased contrast.

Performance comparison of publicly available retinal blood vessel segmentation methods.

Retinal blood vessel structure is an important indicator of many retinal and systemic diseases, which has motivated the development of various image segmentation methods for the blood vessels. In this study, two supervised and three unsupervised segmentation methods with a publicly available implementation are reviewed and quantitatively compared with each other on five public databases with ground truth segmentation of the vessels. Each method is tested under consistent conditions with two types of preprocessing, and the parameters of the methods are optimized for each database. Additionally, possibility to predict the parameters of the methods by the linear regression model is tested for each database. Resolution of the input images and amount of the vessel pixels in the ground truth are used as predictors. The results show the positive influence of preprocessing on the performance of the unsupervised methods. The methods show similar performance for segmentation accuracy, with the best performance achieved by the method by Azzopardi et al. (Acc 94.0) on ARIADB, the method by Soares et al. (Acc 94.6, 94.7) on CHASEDB1 and DRIVE, and the method by Nguyen et al. (Acc 95.8, 95.5) on HRF and STARE. The method by Soares et al. performed better with regard to the area under the ROC curve. Qualitative differences between the methods are discussed. Finally, it was possible to predict the parameter settings that give performance close to the optimized performance of each method.

Optical identification based on time domain optical coherence tomography.

We present a novel method for optical identification, i.e., authenticating valuable documents such as a passport, credit cards, and bank notes, using optical coherence tomography (OCT). An OCT system can capture three-dimensional (3D) images and visualize the internal structure of an object. In our work, as an object, we consider a multilayered optical identification tag composed of a limited number of thin layers (10-100 µm thick). The thickness, width, and location of the layers in the tag encode a unique identification information. Reading of the tag is done using a time domain OCT (TD-OCT) system. Typically, a TD-OCT system requires continuous mechanical scanning in one or more directions to get a 3D volume image of an object. The continuous scanning implies a complicated optical setup, which makes an OCT system fragile and expensive. We propose to avoid the conventional scanning by (1) not requiring 3D imaging, and (2) utilizing the motion of the optical tag itself. The motion is introduced to the tag reader, for example, by a user, which replaces the need for conventional scanning. The absence of a conventional scanning mechanism makes the proposed OCT method very simple and suited for identification purposes; however, it also puts some constraints to the construction of the optical tag, which we discuss in this paper in detail.

Optical absorption spectra of human articular cartilage correlate with biomechanical properties, histological score and biochemical composition.

This study investigates the relationship between the optical response of human articular cartilage in the visible (VIS) and near infrared (NIR) spectral range and its matrix properties.Full-thickness osteochondral cores (dia. = 16 mm, n = 50) were extracted from human cadaver knees (N = 13) at four anatomical locations and divided into quadrants. Absorption spectra were acquired in the spectral range 400-1100 nm from one quadrant. Reference biomechanical, biochemical composition, histological, and cartilage thickness measurements were obtained from two other quadrants. A multivariate statistical technique based on partial least squares (PLS) regression was then employed to investigate the correlation between the absorption spectra and tissue properties.Our results demonstrate that cartilage optical response correlates with its function, composition and morphology, as indicated by the significant relationship between spectral predicted and measured biomechanical (79.0% ⩽ R(2) ⩽ 80.3%, p < 0.0001), biochemical (65.1% ⩽ R(2) ⩽ 81.0%, p < 0.0001), and histological scores ([Formula: see text] = 83.3%, p < 0.0001) properties. Significant correlation was also obtained with the non-calcified cartilage thickness ([Formula: see text] = 83.2%, p < 0.0001).We conclude that optical absorption of human cartilage in the VIS and NIR spectral range correlates with the overall tissue properties, thus providing knowledge that could facilitate development of systems for rapid assessment of tissue integrity.

Eigenvectors of optimal color spectra.

Principal component analysis (PCA) and weighted PCA were applied to spectra of optimal colors belonging to the outer surface of the object-color solid or to so-called MacAdam limits. The correlation matrix formed from this data is a circulant matrix whose biggest eigenvalue is simple and the corresponding eigenvector is constant. All other eigenvalues are double, and the eigenvectors can be expressed with trigonometric functions. Found trigonometric functions can be used as a general basis to reconstruct all possible smooth reflectance spectra. When the spectral data are weighted with an appropriate weight function, the essential part of the color information is compressed to the first three components and the shapes of the first three eigenvectors correspond to one achromatic response function and to two chromatic response functions, the latter corresponding approximately to Munsell opponent-hue directions 9YR-9B and 2BG-2R.

A wide spectral range reflectance and luminescence imaging system.

In this study, we introduce a wide spectral range (200-2500 nm) imaging system with a 250 µm minimum spatial resolution, which can be freely modified for a wide range of resolutions and measurement geometries. The system has been tested for reflectance and luminescence measurements, but can also be customized for transmittance measurements. This study includes the performance results of the developed system, as well as examples of spectral images. Discussion of the system relates it to existing systems and methods. The wide range spectral imaging system that has been developed is however highly customizable and has great potential in many practical applications.

Nondestructive fluorescence-based quantification of threose-induced collagen cross-linking in bovine articular cartilage.

Extensive collagen cross-linking affects the mechanical competence of articular cartilage: it can make the cartilage stiffer and more brittle. The concentrations of the best known cross-links, pyridinoline and pentosidine, can be accurately determined by destructive high-performance liquid chromatography (HPLC). We explore a nondestructive evaluation of cross-linking by using the intrinsic fluorescence of the intact cartilage. Articular cartilage samples from bovine knee joints were incubated in threose solution for 40 and 100 h to increase the collagen cross-linking. Control samples without threose were also prepared. Excitation-emission matrices at wavelengths of 220 to 950 nm were acquired from the samples, and the pentosidine and pyridinoline cross-links and the collagen concentrations were determined using HPLC. After the threose treatment, pentosidine and lysyl pyridinole (LP) concentrations increased. The intrinsic fluorescence, excited below 350 nm, decreased and was related to pentosidine [r = -0.90, 240/325 nm (excitation/emission)] or LP (r = -0.85, 235/285 nm) concentrations. Due to overlapping, the changes in emission could not be linked specifically to the recorded cross-links. However, the fluorescence signal enabled a nondestructive optical estimate of changes in the pentosidine and LP cross-linking of intact articular cartilage.

Number of colors generated by smooth nonfluorescent reflectance spectra.

In this study, we have analyzed statistical properties of the values of the first- and second-order derivatives of spectral reflectance curves. We show that values of all four tested spectral data sets have very similar statistical properties. We set outer limits that bound the clear majority of the values of the first- and second-order derivatives. These limits define smoothness of all nonfluorescent reflectance curves, and they can be used to form a new object color solid inside classical MacAdam limits, including all possible colors generated by smooth nonfluorescent reflectance spectra. We have used the CIELAB color space and filled the new object color solid with a hexagonal closest packing-point lattice to estimate that there exist about 2.5 million different colors, when viewed under the D65 standard illumination.

Highlight detection and removal from spectral image.

We present a constrained spectral unmixing method to remove highlight from a single spectral image. In the constrained spectral unmixing method, the constraints have been imposed so that all the fractions of diffuse and highlight reflection sum up to 1 and are positive. As a result, the spectra of the diffuse image are always positive. The spectral power distribution (SPD) of the light source has been used as the pure highlight spectrum. The pure diffuse spectrum of the measured spectrum has been chosen from the set of diffuse spectra. The pure diffuse spectrum has a minimum angle among the angles calculated between spectra from a set of diffuse spectra and the measured spectrum projected onto the subspace orthogonal to the SPD of the light source. The set of diffuse spectra has been collected by an automated target generation program from the diffuse part in the image. Constrained energy minimization in a finite impulse response linear filter has been used to detect the highlight and diffuse parts in the image. Results by constrained spectral unmixing have been compared with results by the orthogonal subspace projection (OSP) method [Proceedings of International Conference on Pattern Recognition (2006), pp. 812-815] and probabilistic principal component analysis (PPCA) [Proceedings of the 4th WSEAS International Conference on Signal Processing, Robotics and Automation (2005), paper 15]. Constrained spectral unmixing outperforms OSP and PPCA in the visual assessment of the diffuse results. The highlight removal method by constrained spectral unmixing is suitable for spectral images.

Optical spectral reflectance of human articular cartilage - relationships with tissue structure, composition and mechanical properties.

The information from spectral reflectance of articular cartilage has been related to the integrity of the tissue. This study explores more in detail the interrelations between the cartilage composition, structure and mechanical properties, and optical spectral reflectance. Using human osteochondral samples the reflectance spectral images of articular cartilage were captured and analyzed by using CIELAB color space as well as principal component analysis. With both analysis methods statistically significant correlations were observed between the reflectance and histological integrity, as assessed by Mankin scoring, tissue proteoglycan content and dynamic modulus. In thick human cartilage, the reflectance was found to be independent of the cartilage thickness, suggesting negligible influence of the underlying subchondral bone. Based on the present results diagnostically relevant information on cartilage quality can be extracted using optical spectral reflectance measurements. These measurements could be feasible during arthroscopic surgery when more in-depth information of the properties of articular cartilage is needed.

Cluster analysis of infrared spectra of rabbit cortical bone samples during maturation and growth.

Bone consists of an organic and an inorganic matrix. During development, bone undergoes changes in its composition and structure. In this study we apply three different cluster analysis algorithms [K-means (KM), fuzzy C-means (FCM) and hierarchical clustering (HCA)], and discriminant analysis (DA) on infrared spectroscopic data from developing cortical bone with the aim of comparing their ability to correctly classify the samples into different age groups. Cortical bone samples from the mid-diaphysis of the humerus of New Zealand white rabbits from three different maturation stages (newborn (NB), immature (11 days-1 month old), mature (3-6 months old)) were used. Three clusters were obtained by KM, FCM and HCA methods on different spectral regions (amide I, phosphate and carbonate). The newborn samples were well separated (71-100% correct classifications) from the other age groups by all bone components. The mature samples (3-6 months old) were well separated (100%) from those of other age groups by the carbonate spectral region, while by the phosphate and amide I regions some samples were assigned to another group (43-71% correct classifications). The greatest variance in the results for all algorithms was observed in the amide I region. In general, FCM clustering performed better than the other methods, and the overall error was lower. The discriminate analysis results showed that by combining the clustering results from all three spectral regions, the ability to predict the correct age group for all samples increased (from 29-86% to 77-91%). This study is the first to compare several clustering methods on infrared spectra of bone. Fuzzy C-means clustering performed best, and its ability to study the degree of memberships of samples to each cluster might be beneficial in future studies of medical diagnostics.

Optical spectral imaging of degeneration of articular cartilage.

Osteoarthritis (OA) is a common musculoskeletal disorder often diagnosed during arthroscopy. In OA, visual color changes of the articular cartilage surface are typically observed. We demonstrate in vitro the potential of visible light spectral imaging (420 to 720 nm) to quantificate these color changes. Intact bovine articular cartilage samples (n=26) are degraded both enzymatically using the collagenase and mechanically using the emery paper (P60 grit, 269 microm particle size). Spectral images are analyzed by using standard CIELAB color coordinates and the principal component analysis (PCA). After collagenase digestion, changes in the CIELAB coordinates and projection of the spectra to PCA eigenvector are statistically significant (p<0.05). After mechanical degradation, the grinding tracks could not be visualized in the RGB presentation, i.e., in the visual appearance of the sample to the naked eye under the D65 illumination. However, after projecting to the chosen eigenvector, the grinding tracks are revealed. The tracks are also seen by using only one wavelength, i.e., 469 nm, however, the contrast in the projection image is 1.6 to 2.5 times higher. Our results support the idea that the spectral imaging can be used for evaluation of the integrity of the cartilage surface.

Evaluation and unification of some methods for estimating reflectance spectra from RGB images.

The problem of estimating spectral reflectances from the responses of a digital camera has received considerable attention recently. This problem can be cast as a regularized regression problem or as a statistical inversion problem. We discuss some previously suggested estimation methods based on critically undersampled RGB measurements and describe some relations between them. We concentrate mainly on those models that are using a priori information in the form of high-resolution measurements. We use the "kernel machine" framework in our evaluations and concentrate on the use of multiple illuminations and on the investigation of the performance of global and locally adapted estimation methods. We also introduce a nonlinear transformation of reflectance values to ensure that the estimated reflection spectra fulfill physically motivated boundary conditions. The reported experimental results are derived from measured and simulated camera responses from the Munsell Matte, NCS, and Pantone data sets.

Regularized learning framework in the estimation of reflectance spectra from camera responses.

For digital cameras, device-dependent pixel values describe the camera's response to the incoming spectrum of light. We convert device-dependent RGB values to device- and illuminant-independent reflectance spectra. Simple regularization methods with widely used polynomial modeling provide an efficient approach for this conversion. We also introduce a more general framework for spectral estimation: regularized least-squares regression in reproducing kernel Hilbert spaces (RKHS). Obtained results show that the regularization framework provides an efficient approach for enhancing the generalization properties of the models.