Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Models.

During image editing, existing deep generative models tend to re-synthesize the entire output from scratch, including the unedited regions. This leads to a significant waste of computation, especially for minor editing operations. In this work, we present Spatially Sparse Inference (SSI), a general-purpose technique that selectively performs computation for edited regions and accelerates various generative models, including both conditional GANs and diffusion models. Our key observation is that users prone to gradually edit the input image. This motivates us to cache and reuse the feature maps of the original image. Given an edited image, we sparsely apply the convolutional filters to the edited regions while reusing the cached features for the unedited areas. Based on our algorithm, we further propose Sparse Incremental Generative Engine (SIGE) to convert the computation reduction to latency reduction on off-the-shelf hardware. With about 1%-area edits, SIGE accelerates DDPM by 3.0× on NVIDIA RTX 3090 and 4.6× on Apple M1 Pro GPU, Stable Diffusion by 7.2× on 3090, and GauGAN by 5.6× on 3090 and 5.2× on M1 Pro GPU. Compared to our conference paper, we enhance SIGE to accommodate attention layers and apply it to Stable Diffusion. Additionally, we offer support for Apple M1 Pro GPU and include more results to substantiate the efficacy of our method.

HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution.

Genomic (DNA) sequences encode an enormous amount of information for gene regulation and protein synthesis. Similar to natural language models, researchers have proposed foundation models in genomics to learn generalizable features from unlabeled genome data that can then be fine-tuned for downstream tasks such as identifying regulatory elements. Due to the quadratic scaling of attention, previous Transformer-based genomic models have used 512 to 4k tokens as context (<0.001% of the human genome), significantly limiting the modeling of long-range interactions in DNA. In addition, these methods rely on tokenizers or fixed k-mers to aggregate meaningful DNA units, losing single nucleotide resolution where subtle genetic variations can completely alter protein function via single nucleotide polymorphisms (SNPs). Recently, Hyena, a large language model based on implicit convolutions was shown to match attention in quality while allowing longer context lengths and lower time complexity. Leveraging Hyena's new long-range capabilities, we present HyenaDNA, a genomic foundation model pretrained on the human reference genome with context lengths of up to 1 million tokens at the single nucleotide-level - an up to 500x increase over previous dense attention-based models. HyenaDNA scales sub-quadratically in sequence length (training up to 160x faster than Transformer), uses single nucleotide tokens, and has full global context at each layer. We explore what longer context enables - including the first use of in-context learning in genomics. On fine-tuned benchmarks from the Nucleotide Transformer, HyenaDNA reaches state-of-the-art (SotA) on 12 of 18 datasets using a model with orders of magnitude less parameters and pretraining data. On the GenomicBenchmarks, HyenaDNA surpasses SotA on 7 of 8 datasets on average by +10 accuracy points. Code at https://github.com/HazyResearch/hyena-dna.

Scalable deep learning to identify brick kilns and aid regulatory capacity.

Improving compliance with environmental regulations is critical for promoting clean environments and healthy populations. In South Asia, brick manufacturing is a major source of pollution but is dominated by small-scale, informal producers who are difficult to monitor and regulate-a common challenge in low-income settings. We demonstrate a low-cost, scalable approach for locating brick kilns in high-resolution satellite imagery from Bangladesh. Our approach identifies kilns with 94.2% accuracy and 88.7% precision and extracts the precise GPS coordinates of every brick kiln across Bangladesh. Using these estimates, we show that at least 12% of the population of Bangladesh (>18 million people) live within 1 km of a kiln and that 77% and 9% of kilns are (illegally) within 1 km of schools and health facilities, respectively. Finally, we show how kilns contribute up to 20.4 µg/[Formula: see text] of [Formula: see text] (particulate matter of a diameter less than 2.5 µm) in Dhaka when the wind blows from an unfavorable direction. We document inaccuracies and potential bias with respect to local regulations in the government data. Our approach demonstrates how machine learning and Earth observation can be combined to better understand the extent and implications of regulatory compliance in informal industry.

Using satellite imagery to understand and promote sustainable development.

Accurate and comprehensive measurements of a range of sustainable development outcomes are fundamental inputs into both research and policy. We synthesize the growing literature that uses satellite imagery to understand these outcomes, with a focus on approaches that combine imagery with machine learning. We quantify the paucity of ground data on key human-related outcomes and the growing abundance and improving resolution (spatial, temporal, and spectral) of satellite imagery. We then review recent machine learning approaches to model-building in the context of scarce and noisy training data, highlighting how this noise often leads to incorrect assessment of model performance. We quantify recent model performance across multiple sustainable development domains, discuss research and policy applications, explore constraints to future progress, and highlight research directions for the field.

Using publicly available satellite imagery and deep learning to understand economic well-being in Africa.

Accurate and comprehensive measurements of economic well-being are fundamental inputs into both research and policy, but such measures are unavailable at a local level in many parts of the world. Here we train deep learning models to predict survey-based estimates of asset wealth across ~ 20,000 African villages from publicly-available multispectral satellite imagery. Models can explain 70% of the variation in ground-measured village wealth in countries where the model was not trained, outperforming previous benchmarks from high-resolution imagery, and comparison with independent wealth measurements from censuses suggests that errors in satellite estimates are comparable to errors in existing ground data. Satellite-based estimates can also explain up to 50% of the variation in district-aggregated changes in wealth over time, with daytime imagery particularly useful in this task. We demonstrate the utility of satellite-based estimates for research and policy, and demonstrate their scalability by creating a wealth map for Africa's most populous country.

Closed-loop optimization of fast-charging protocols for batteries with machine learning.

Simultaneously optimizing many design parameters in time-consuming experiments causes bottlenecks in a broad range of scientific and engineering disciplines1,2. One such example is process and control optimization for lithium-ion batteries during materials selection, cell manufacturing and operation. A typical objective is to maximize battery lifetime; however, conducting even a single experiment to evaluate lifetime can take months to years3-5. Furthermore, both large parameter spaces and high sampling variability3,6,7 necessitate a large number of experiments. Hence, the key challenge is to reduce both the number and the duration of the experiments required. Here we develop and demonstrate a machine learning methodology  to efficiently optimize a parameter space specifying the current and voltage profiles of six-step, ten-minute fast-charging protocols for maximizing battery cycle life, which can alleviate range anxiety for electric-vehicle users8,9. We combine two key elements to reduce the optimization cost: an early-prediction model5, which reduces the time per experiment by predicting the final cycle life using data from the first few cycles, and a Bayesian optimization algorithm10,11, which reduces the number of experiments by balancing exploration and exploitation to efficiently probe the parameter space of charging protocols. Using this methodology, we rapidly identify high-cycle-life charging protocols among 224 candidates in 16 days (compared with over 500 days using exhaustive search without early prediction), and subsequently validate the accuracy and efficiency of our optimization approach. Our closed-loop methodology automatically incorporates feedback from past experiments to inform future decisions and can be generalized to other applications in battery design and, more broadly, other scientific domains that involve time-intensive experiments and multi-dimensional design spaces.

Rapid identification of pathogenic bacteria using Raman spectroscopy and deep learning.

Raman optical spectroscopy promises label-free bacterial detection, identification, and antibiotic susceptibility testing in a single step. However, achieving clinically relevant speeds and accuracies remains challenging due to weak Raman signal from bacterial cells and numerous bacterial species and phenotypes. Here we generate an extensive dataset of bacterial Raman spectra and apply deep learning approaches to accurately identify 30 common bacterial pathogens. Even on low signal-to-noise spectra, we achieve average isolate-level accuracies exceeding 82% and antibiotic treatment identification accuracies of 97.0±0.3%. We also show that this approach distinguishes between methicillin-resistant and -susceptible isolates of Staphylococcus aureus (MRSA and MSSA) with 89±0.1% accuracy. We validate our results on clinical isolates from 50 patients. Using just 10 bacterial spectra from each patient isolate, we achieve treatment identification accuracies of 99.7%. Our approach has potential for culture-free pathogen identification and antibiotic susceptibility testing, and could be readily extended for diagnostics on blood, urine, and sputum.

Using machine learning to discover shape descriptors for predicting emulsion stability in a microfluidic channel.

In soft matter consisting of many deformable objects, object shapes often carry important information about local forces and their interactions with the local environment, and can be tightly coupled to the bulk properties and functions. In a concentrated emulsion, for example, the shapes of individual droplets are directly related to the local stress arising from interactions with neighboring drops, which in turn determine their stability and the resulting rheological properties. Shape descriptors used in prior work on single drops and dilute emulsions, where droplet-droplet interactions are largely negligible and the drop shapes are simple, are insufficient to fully capture the broad range of droplet shapes in a concentrated system. This paper describes the application of a machine learning method, specifically a convolutional autoencoder model, that learns to: (1) discover a low-dimensional code (8-dimensional) to describe droplet shapes within a concentrated emulsion, and (2) predict whether the drop will become unstable and undergo break-up. The input consists of images (N = 500 002) of two-dimensional droplet boundaries extracted from movies of a concentrated emulsion flowing through a confined microfluidic channel as a monolayer. The model is able to faithfully reconstruct droplet shapes, as well as to achieve a classification accuracy of 91.7% in the prediction of droplet break-up, compared with ∼60% using conventional scalar descriptors based on droplet elongation. It is observed that 4 out of the 8 dimensions of the code are interpretable, corresponding to drop skewness, elongation, throat size, and surface curvature, respectively. Furthermore, the results show that drop elongation, throat size, and surface curvature are dominant factors in predicting droplet break-up for the flow conditions tested. The method presented is expected to facilitate follow-on work to identify the relationship between drop shapes and the interactions with other drops, and to identify potentially new modes of break-up mechanisms in a concentrated system. Finally, the method developed here should also apply to other soft materials such as foams, gels, and cells and tissues.

Coupling between oxygen redox and cation migration explains unusual electrochemistry in lithium-rich layered oxides.

Lithium-rich layered transition metal oxide positive electrodes offer access to anion redox at high potentials, thereby promising high energy densities for lithium-ion batteries. However, anion redox is also associated with several unfavorable electrochemical properties, such as open-circuit voltage hysteresis. Here we reveal that in Li1.17-x Ni0.21Co0.08Mn0.54O2, these properties arise from a strong coupling between anion redox and cation migration. We combine various X-ray spectroscopic, microscopic, and structural probes to show that partially reversible transition metal migration decreases the potential of the bulk oxygen redox couple by > 1 V, leading to a reordering in the anionic and cationic redox potentials during cycling. First principles calculations show that this is due to the drastic change in the local oxygen coordination environments associated with the transition metal migration. We propose that this mechanism is involved in stabilizing the oxygen redox couple, which we observe spectroscopically to persist for 500 charge/discharge cycles.

Unsupervised Data Mining in nanoscale X-ray Spectro-Microscopic Study of NdFeB Magnet.

Novel developments in X-ray based spectro-microscopic characterization techniques have increased the rate of acquisition of spatially resolved spectroscopic data by several orders of magnitude over what was possible a few years ago. This accelerated data acquisition, with high spatial resolution at nanoscale and sensitivity to subtle differences in chemistry and atomic structure, provides a unique opportunity to investigate hierarchically complex and structurally heterogeneous systems found in functional devices and materials systems. However, handling and analyzing the large volume data generated poses significant challenges. Here we apply an unsupervised data-mining algorithm known as DBSCAN to study a rare-earth element based permanent magnet material, Nd2Fe14B. We are able to reduce a large spectro-microscopic dataset of over 300,000 spectra to 3, preserving much of the underlying information. Scientists can easily and quickly analyze in detail three characteristic spectra. Our approach can rapidly provide a concise representation of a large and complex dataset to materials scientists and chemists. For example, it shows that the surface of common Nd2Fe14B magnet is chemically and structurally very different from the bulk, suggesting a possible surface alteration effect possibly due to the corrosion, which could affect the material's overall properties.

Combining satellite imagery and machine learning to predict poverty.

Reliable data on economic livelihoods remain scarce in the developing world, hampering efforts to study these outcomes and to design policies that improve them. Here we demonstrate an accurate, inexpensive, and scalable method for estimating consumption expenditure and asset wealth from high-resolution satellite imagery. Using survey and satellite data from five African countries--Nigeria, Tanzania, Uganda, Malawi, and Rwanda--we show how a convolutional neural network can be trained to identify image features that can explain up to 75% of the variation in local-level economic outcomes. Our method, which requires only publicly available data, could transform efforts to track and target poverty in developing countries. It also demonstrates how powerful machine learning techniques can be applied in a setting with limited training data, suggesting broad potential application across many scientific domains.