mHealth hyperspectral learning for instantaneous spatiospectral imaging of hemodynamics.

Hyperspectral imaging acquires data in both the spatial and frequency domains to offer abundant physical or biological information. However, conventional hyperspectral imaging has intrinsic limitations of bulky instruments, slow data acquisition rate, and spatiospectral trade-off. Here we introduce hyperspectral learning for snapshot hyperspectral imaging in which sampled hyperspectral data in a small subarea are incorporated into a learning algorithm to recover the hypercube. Hyperspectral learning exploits the idea that a photograph is more than merely a picture and contains detailed spectral information. A small sampling of hyperspectral data enables spectrally informed learning to recover a hypercube from a red-green-blue (RGB) image without complete hyperspectral measurements. Hyperspectral learning is capable of recovering full spectroscopic resolution in the hypercube, comparable to high spectral resolutions of scientific spectrometers. Hyperspectral learning also enables ultrafast dynamic imaging, leveraging ultraslow video recording in an off-the-shelf smartphone, given that a video comprises a time series of multiple RGB images. To demonstrate its versatility, an experimental model of vascular development is used to extract hemodynamic parameters via statistical and deep learning approaches. Subsequently, the hemodynamics of peripheral microcirculation is assessed at an ultrafast temporal resolution up to a millisecond, using a conventional smartphone camera. This spectrally informed learning method is analogous to compressed sensing; however, it further allows for reliable hypercube recovery and key feature extractions with a transparent learning algorithm. This learning-powered snapshot hyperspectral imaging method yields high spectral and temporal resolutions and eliminates the spatiospectral trade-off, offering simple hardware requirements and potential applications of various machine learning techniques.

Edible Matrix Code with Photogenic Silk Proteins.

Counterfeit medicines are a healthcare security problem, posing not only a direct threat to patient safety and public health but also causing heavy economic losses. Current anticounterfeiting methods are limited due to the toxicity of the constituent materials and the focus of secondary packaging level protections. We introduce an edible, imperceptible, and scalable matrix code of information representation and data storage for pharmaceutical products. This matrix code is digestible as it is composed of silk fibroin genetically encoded with fluorescent proteins produced by ecofriendly, sustainable silkworm farming. Three distinct fluorescence emission colors are incorporated into a multidimensional parameter space with a variable encoding capacity in a format of matrix arrays. This code is smartphone-readable to extract a digitized security key augmented by a deep neural network for overcoming fabrication imperfections and a cryptographic hash function for enhanced security. The biocompatibility, photostability, thermal stability, long-term reliability, and low bit error ratio of the code support the immediate feasibility for dosage-level anticounterfeit measures and authentication features. The edible code affixed to each medicine can serve as serialization, track and trace, and authentication at the dosage level, empowering every patient to play a role in combating illicit pharmaceuticals.

Compressive recovery of smartphone RGB spectral sensitivity functions.

Spectral response (or sensitivity) functions of a three-color image sensor (or trichromatic camera) allow a mapping from spectral stimuli to RGB color values. Like biological photosensors, digital RGB spectral responses are device dependent and significantly vary from model to model. Thus, the information on the RGB spectral response functions of a specific device is vital in a variety of computer vision as well as mobile health (mHealth) applications. Theoretically, spectral response functions can directly be measured with sophisticated calibration equipment in a specialized laboratory setting, which is not easily accessible for most application developers. As a result, several mathematical methods have been proposed relying on standard color references. Typical optimization frameworks with constraints are often complicated, requiring a large number of colors. We report a compressive sensing framework in the frequency domain for accurately predicting RGB spectral response functions only with several primary colors. Using a scientific camera, we first validate the estimation method with direct spectral sensitivity measurements and ensure that the root mean square errors between the ground truth and recovered RGB spectral response functions are negligible. We further recover the RGB spectral response functions of smartphones and validate with an expanded color checker reference. We expect that this simple yet reliable estimation method of RGB spectral sensitivity can easily be applied for color calibration and standardization in machine vision, hyperspectral filters, and mHealth applications that capitalize on the built-in cameras of smartphones.

A Pearl Spectrometer.

Information recovery from incomplete measurements, typically performed by a numerical means, is beneficial in a variety of classical and quantum signal processing. Random and sparse sampling with nanophotonic and light scattering approaches has received attention to overcome the hardware limitations of conventional spectrometers and hyperspectral imagers but requires high-precision nanofabrications and bulky media. We report a simple spectral information processing scheme in which light transport through an Anderson-localized medium serves as an entropy source for compressive sampling directly in the frequency domain. As implied by the "lustrous" reflection originating from the exquisite multilayered nanostructures, a pearl (or mother-of-pearl) allows us to exploit the spatial and spectral intensity fluctuations originating from strong light localization for extracting salient spectral information with a compact and thin form factor. Pearl-inspired light localization in low-dimensional structures can offer an alternative of spectral information processing by hybridizing digital and physical properties at a material level.

mHealth spectroscopy of blood hemoglobin with spectral super-resolution.

Although blood hemoglobin (Hgb) testing is a routine procedure in a variety of clinical situations, noninvasive, continuous, and real-time blood Hgb measurements are still challenging. Optical spectroscopy can offer noninvasive blood Hgb quantification, but requires bulky optical components that intrinsically limit the development of mobile health (mHealth) technologies. Here, we report spectral super-resolution (SSR) spectroscopy that virtually transforms the built-in camera (RGB sensor) of a smartphone into a hyperspectral imager for accurate and precise blood Hgb analyses. Statistical learning of SSR enables us to reconstruct detailed spectra from three color RGB data. Peripheral tissue imaging with a mobile application is further combined to compute exact blood Hgb content without a priori personalized calibration. Measurements over a wide range of blood Hgb values show reliable performance of SSR blood Hgb quantification. Given that SSR does not require additional hardware accessories, the mobility, simplicity, and affordability of conventional smartphones support the idea that SSR blood Hgb measurements can be used as an mHealth method.