SpectralGPT: Spectral Remote Sensing Foundation Model.

The foundation model has recently garnered significant attention due to its potential to revolutionize the field of visual representation learning in a self-supervised manner. While most foundation models are tailored to effectively process RGB images for various visual tasks, there is a noticeable gap in research focused on spectral data, which offers valuable information for scene understanding, especially in remote sensing (RS) applications. To fill this gap, we created for the first time a universal RS foundation model, named SpectralGPT, which is purpose-built to handle spectral RS images using a novel 3D generative pretrained transformer (GPT). Compared to existing foundation models, SpectralGPT 1) accommodates input images with varying sizes, resolutions, time series, and regions in a progressive training fashion, enabling full utilization of extensive RS Big Data; 2) leverages 3D token generation for spatial-spectral coupling; 3) captures spectrally sequential patterns via multi-target reconstruction; 4) trains on one million spectral RS images, yielding models with over 600 million parameters. Our evaluation highlights significant performance improvements with pretrained SpectralGPT models, signifying substantial potential in advancing spectral RS Big Data applications within the field of geoscience across four downstream tasks: single/multi-label scene classification, semantic segmentation, and change detection.

Comprehensive mapping of lunar surface chemistry by adding Chang'e-5 samples with deep learning.

Lunar surface chemistry is essential for revealing petrological characteristics to understand the evolution of the Moon. Existing chemistry mapping from Apollo and Luna returned samples could only calibrate chemical features before 3.0 Gyr, missing the critical late period of the Moon. Here we present major oxides chemistry maps by adding distinctive 2.0 Gyr Chang'e-5 lunar soil samples in combination with a deep learning-based inversion model. The inferred chemical contents are more precise than the Lunar Prospector Gamma-Ray Spectrometer (GRS) maps and are closest to returned samples abundances compared to existing literature. The verification of in situ measurement data acquired by Chang'e 3 and Chang'e 4 lunar rover demonstrated that Chang'e-5 samples are indispensable ground truth in mapping lunar surface chemistry. From these maps, young mare basalt units are determined which can be potential sites in future sample return mission to constrain the late lunar magmatic and thermal history.

Iterative Training Sample Augmentation for Enhancing Land Cover Change Detection Performance With Deep Learning Neural Network.

Labeled samples are important in achieving land cover change detection (LCCD) tasks via deep learning techniques with remote sensing images. However, labeling samples for change detection with bitemporal remote sensing images is labor-intensive and time-consuming. Moreover, manually labeling samples between bitemporal images requires professional knowledge for practitioners. To address this problem in this article, an iterative training sample augmentation (ITSA) strategy to couple with a deep learning neural network for improving LCCD performance is proposed here. In the proposed ITSA, we start by measuring the similarity between an initial sample and its four-quarter-overlapped neighboring blocks. If the similarity satisfies a predefined constraint, then a neighboring block will be selected as the potential sample. Next, a neural network is trained with renewed samples and used to predict an intermediate result. Finally, these operations are fused into an iterative algorithm to achieve the training and prediction of a neural network. The performance of the proposed ITSA strategy is verified with some widely used change detection deep learning networks using seven pairs of real remote sensing images. The excellent visual performance and quantitative comparisons from the experiments clearly indicate that detection accuracies of LCCD can be effectively improved when a deep learning network is coupled with the proposed ITSA. For example, compared with some state-of-the-art methods, the quantitative improvement is 0.38%-7.53% in terms of overall accuracy. Moreover, the improvement is robust, generic to both homogeneous and heterogeneous images, and universally adaptive to various neural networks of LCCD. The code will be available at https://github.com/ImgSciGroup/ITSA.

DRBM-ClustNet: A Deep Restricted Boltzmann-Kohonen Architecture for Data Clustering.

A Bayesian deep restricted Boltzmann-Kohonen architecture for data clustering termed deep restricted Boltzmann machine (DRBM)-ClustNet is proposed. This core-clustering engine consists of a DRBM for processing unlabeled data by creating new features that are uncorrelated and have large variance with each other. Next, the number of clusters is predicted using the Bayesian information criterion (BIC), followed by a Kohonen network (KN)-based clustering layer. The processing of unlabeled data is done in three stages for efficient clustering of the nonlinearly separable datasets. In the first stage, DRBM performs nonlinear feature extraction by capturing the highly complex data representation by projecting the feature vectors of d dimensions into n dimensions. Most clustering algorithms require the number of clusters to be decided a priori; hence, here, to automate the number of clusters in the second stage, we use BIC. In the third stage, the number of clusters derived from BIC forms the input for the KN, which performs clustering of the feature-extracted data obtained from the DRBM. This method overcomes the general disadvantages of clustering algorithms, such as the prior specification of the number of clusters, convergence to local optima, and poor clustering accuracy on nonlinear datasets. In this research, we use two synthetic datasets, 15 benchmark datasets from the UCI Machine Learning repository, and four image datasets to analyze the DRBM-ClustNet. The proposed framework is evaluated based on clustering accuracy and ranked against other state-of-the-art clustering methods. The obtained results demonstrate that the DRBM-ClustNet outperforms state-of-the-art clustering algorithms.

A novel optimal multi-pattern matching method with wildcards for DNA sequence.

BACKGROUND: DNA sequence alignment is one of the most fundamental and important operation to identify which gene family may contain this sequence, pattern matching for DNA sequence has been a fundamental issue in biomedical engineering, biotechnology and health informatics. OBJECTIVE: To solve this problem, this study proposes an optimal multi pattern matching with wildcards for DNA sequence. METHODS: This proposed method packs the patterns and a sliding window of texts, and the window slides along the given packed text, matching against stored packed patterns. RESULTS: Three data sets are used to test the performance of the proposed algorithm, and the algorithm was seen to be more efficient than the competitors because its operation is close to machine language. CONCLUSIONS: Theoretical analysis and experimental results both demonstrate that the proposed method outperforms the state-of-the-art methods and is especially effective for the DNA sequence.

Scheduling-Guided Automatic Processing of Massive Hyperspectral Image Classification on Cloud Computing Architectures.

The large data volume and high algorithm complexity of hyperspectral image (HSI) problems have posed big challenges for efficient classification of massive HSI data repositories. Recently, cloud computing architectures have become more relevant to address the big computational challenges introduced in the HSI field. This article proposes an acceleration method for HSI classification that relies on scheduling metaheuristics to automatically and optimally distribute the workload of HSI applications across multiple computing resources on a cloud platform. By analyzing the procedure of a representative classification method, we first develop its distributed and parallel implementation based on the MapReduce mechanism on Apache Spark. The subtasks of the processing flow that can be processed in a distributed way are identified as divisible tasks. The optimal execution of this application on Spark is further formulated as a divisible scheduling framework that takes into account both task execution precedences and task divisibility when allocating the divisible and indivisible subtasks onto computing nodes. The formulated scheduling framework is an optimization procedure that searches for optimized task assignments and partition counts for divisible tasks. Two metaheuristic algorithms are developed to solve this divisible scheduling problem. The scheduling results provide an optimized solution to the automatic processing of HSI big data on clouds, improving the computational efficiency of HSI classification by exploring the parallelism during the parallel processing flow. Experimental results demonstrate that our scheduling-guided approach achieves remarkable speedups by facilitating the automatic processing of HSI classification on Spark, and is scalable to the increasing HSI data volume.

Lunar impact crater identification and age estimation with Chang'E data by deep and transfer learning.

Impact craters, which can be considered the lunar equivalent of fossils, are the most dominant lunar surface features and record the history of the Solar System. We address the problem of automatic crater detection and age estimation. From initially small numbers of recognized craters and dated craters, i.e., 7895 and 1411, respectively, we progressively identify new craters and estimate their ages with Chang'E data and stratigraphic information by transfer learning using deep neural networks. This results in the identification of 109,956 new craters, which is more than a dozen times greater than the initial number of recognized craters. The formation systems of 18,996 newly detected craters larger than 8 km are estimated. Here, a new lunar crater database for the mid- and low-latitude regions of the Moon is derived and distributed to the planetary community together with the related data analysis.

Automatic Attribute Profiles.

Morphological attribute profiles are multilevel decompositions of images obtained with a sequence of transformations performed by connected operators. They have been extensively employed in performing multi-scale and region-based analysis in a large number of applications. One main, still unresolved, issue is the selection of filter parameters able to provide representative and non-redundant threshold decomposition of the image. This paper presents a framework for the automatic selection of filter thresholds based on Granulometric Characteristic Functions (GCFs). GCFs describe the way that non-linear morphological filters simplify a scene according to a given measure. Since attribute filters rely on a hierarchical representation of an image (e.g., the Tree of Shapes) for their implementation, GCFs can be efficiently computed by taking advantage of the tree representation. Eventually, the study of the GCFs allows the identification of a meaningful set of thresholds. Therefore, a trial and error approach is not necessary for the threshold selection, automating the process and in turn decreasing the computational time. It is shown that the redundant information is reduced within the resulting profiles (a problem of high occurrence, as regards manual selection). The proposed approach is tested on two real remote sensing data sets, and the classification results are compared with strategies present in the literature.

Adaptive Markov random fields for joint unmixing and segmentation of hyperspectral images.

Linear spectral unmixing is a challenging problem in hyperspectral imaging that consists of decomposing an observed pixel into a linear combination of pure spectra (or endmembers) with their corresponding proportions (or abundances). Endmember extraction algorithms can be employed for recovering the spectral signatures while abundances are estimated using an inversion step. Recent works have shown that exploiting spatial dependencies between image pixels can improve spectral unmixing. Markov random fields (MRF) are classically used to model these spatial correlations and partition the image into multiple classes with homogeneous abundances. This paper proposes to define the MRF sites using similarity regions. These regions are built using a self-complementary area filter that stems from the morphological theory. This kind of filter divides the original image into flat zones where the underlying pixels have the same spectral values. Once the MRF has been clearly established, a hierarchical Bayesian algorithm is proposed to estimate the abundances, the class labels, the noise variance, and the corresponding hyperparameters. A hybrid Gibbs sampler is constructed to generate samples according to the corresponding posterior distribution of the unknown parameters and hyperparameters. Simulations conducted on synthetic and real AVIRIS data demonstrate the good performance of the algorithm.

Spectral-spatial classification of hyperspectral data based on a stochastic minimum spanning forest approach.

In this paper, a new method for supervised hyperspectral data classification is proposed. In particular, the notion of stochastic minimum spanning forest (MSF) is introduced. For a given hyperspectral image, a pixelwise classification is first performed. From this classification map, M marker maps are generated by randomly selecting pixels and labeling them as markers for the construction of MSFs. The next step consists in building an MSF from each of the M marker maps. Finally, all the M realizations are aggregated with a maximum vote decision rule in order to build the final classification map. The proposed method is tested on three different data sets of hyperspectral airborne images with different resolutions and contexts. The influences of the number of markers and of the number of realizations M on the results are investigated in experiments. The performance of the proposed method is compared to several classification techniques (both pixelwise and spectral-spatial) using standard quantitative criteria and visual qualitative evaluation.

Segmentation and classification of hyperspectral images using minimum spanning forest grown from automatically selected markers.

A new method for segmentation and classification of hyperspectral images is proposed. The method is based on the construction of a minimum spanning forest (MSF) from region markers. Markers are defined automatically from classification results. For this purpose, pixelwise classification is performed, and the most reliable classified pixels are chosen as markers. Each classification-derived marker is associated with a class label. Each tree in the MSF grown from a marker forms a region in the segmentation map. By assigning a class of each marker to all the pixels within the region grown from this marker, a spectral-spatial classification map is obtained. Furthermore, the classification map is refined using the results of a pixelwise classification and a majority voting within the spatially connected regions. Experimental results are presented for three hyperspectral airborne images. The use of different dissimilarity measures for the construction of the MSF is investigated. The proposed scheme improves classification accuracies, when compared to previously proposed classification techniques, and provides accurate segmentation and classification maps.

Glaucoma filtration surgery and retinal oxygen saturation.

PURPOSE: Glaucoma may involve disturbances in retinal oxygenation and blood flow. The purpose of this study was to measure the effect of glaucoma filtration surgery on retinal vessel oxygen saturation. METHODS: A noninvasive spectrophotometric retinal oximeter was used to measure hemoglobin oxygen saturation in retinal arterioles and venules before and after glaucoma filtration surgery. Twenty-five consecutive patients were recruited, and 19 had adequate image quality. Fourteen underwent trabeculectomy and five glaucoma tube surgery. Twelve had primary open-angle glaucoma and seven had exfoliative glaucoma. IOP decreased from 23 +/- 7 to 10 +/- 4 mm Hg (mean +/- SD, P = 0.0001). RESULTS: Oxygen saturation increased in retinal arterioles from 97% +/- 4% to 99% +/- 6% (n = 19; P = 0.046) after surgery and was unchanged in venules (63% +/- 5% before surgery and 64% +/- 6% after, P = 0.76). There were no significant changes in saturation in the fellow eyes (P > 0.60). The arteriovenous difference was 34% before and 36% after surgery (P = 0.35). CONCLUSIONS: Glaucoma filtration surgery had almost no effect on retinal vessel oxygen saturation.

Oxygen saturation in human retinal vessels is higher in dark than in light.

PURPOSE: Animal studies have indicated that retinal oxygen consumption is greater in dark than light. In this study, oxygen saturation is measured in retinal vessels of healthy humans during dark and light. METHODS: The oximeter consists of a fundus camera, a beam splitter, a digital camera and software, which calculates hemoglobin oxygen saturation in the retinal vessels. In the first experiment, 18 healthy individuals underwent oximetry measurements after 30 minutes in the dark, followed by alternating 5-minute periods of white light (80 cd/m(2)) and dark. In the second experiment, 23 volunteers underwent oximetry measurements after 30 minutes in the dark, followed by light at 1, 10, and 100 cd/m(2). Three subjects were excluded from analysis in the first experiment and four in the second experiment because of poor image quality. RESULTS: In the first experiment, the arteriolar saturation decreased from 92% +/- 4% (n = 15; mean +/- SD) after 30 minutes in the dark to 89% +/- 5% after 5 minutes in the light (P = 0.008). Corresponding numbers for venules are 60% +/- 5% in the dark and 55% +/- 10% (P = 0.020) in the light. In the second experiment, the arteriolar saturation was 92% +/- 4% in the dark and 88% +/- 7% in 100 cd/m(2) light (n = 19, P = 0.012). The corresponding values for venules were 59% +/- 9% in the dark and 55% +/- 10% in 100 cd/m(2) light (P = 0.065). CONCLUSIONS: Oxygen saturation in retinal blood vessels is higher in dark than in 80 or 100 cd/m(2) light in human retinal arterioles and venules. The authors propose that this is a consequence of increased oxygen demand in the outer retina in the dark.

Automatic retinal oximetry.

PURPOSE: To measure hemoglobin oxygen saturation (SO(2)) in retinal vessels and to test the reproducibility and sensitivity of an automatic spectrophotometric oximeter. METHODS: Specialized software automatically identifies the retinal blood vessels on fundus images, which are obtained with four different wavelengths of light. The software calculates optical density ratios (ODRs) for each vessel. The reproducibility was evaluated by analyzing five repeated measurements of the same vessels. A linear relationship between SO(2) and ODR was assumed and a linear model derived. After calibration, reproducibility and sensitivity were calculated in terms of SO(2). Systemic hyperoxia (n = 16) was induced in healthy volunteers by changing the O(2) concentration in inhaled air from 21% to 100%. RESULTS: The automatic software enhanced reproducibility, and the mean SD for repeated measurements was 3.7% for arterioles and 5.3% venules, in terms of percentage of SO(2) (five repeats, 10 individuals). The model derived for calibration was SO(2) = 125 - 142 . ODR. The arterial SO(2) measured 96% +/- 9% (mean +/- SD) during normoxia and 101% +/- 8% during hyperoxia (n = 16). The difference between normoxia and hyperoxia was significant (P = 0.0027, paired t-test). Corresponding numbers for venules were 55% +/- 14% and 78% +/- 15% (P < 0.0001). SO(2) is displayed as a pseudocolor map drawn on fundus images. CONCLUSIONS: The retinal oximeter is reliable, easy to use, and sensitive to changes in SO(2) when concentration of O(2) in inhaled air is changed.