Silent Speech Recognition with Strain Sensors and Deep Learning Analysis of Directional Facial Muscle Movement.

Silent communication based on biosignals from facial muscle requires accurate detection of its directional movement and thus optimally positioning minimum numbers of sensors for higher accuracy of speech recognition with a minimal person-to-person variation. So far, previous approaches based on electromyogram or pressure sensors are ineffective in detecting the directional movement of facial muscles. Therefore, in this study, high-performance strain sensors are used for separately detecting x- and y-axis strain. Directional strain distribution data of facial muscle is obtained by applying three-dimensional digital image correlation. Deep learning analysis is utilized for identifying optimal positions of directional strain sensors. The recognition system with four directional strain sensors conformably attached to the face shows silent vowel recognition with 85.24% accuracy and even 76.95% for completely nonobserved subjects. These results show that detection of the directional strain distribution at the optimal facial points will be the key enabling technology for highly accurate silent speech recognition.

Tough-interface-enabled stretchable electronics using non-stretchable polymer semiconductors and conductors.

Semiconducting polymer thin films are essential elements of soft electronics for both wearable and biomedical applications1-11. However, high-mobility semiconducting polymers are usually brittle and can be easily fractured under small strains (<10%)12-14. Recently, the improved intrinsic mechanical properties of semiconducting polymer films have been reported through molecular design15-18 and nanoconfinement19. Here we show that engineering the interfacial properties between a semiconducting thin film and a substrate can notably delay microcrack formation in the film. We present a universal design strategy that involves covalently bonding a dissipative interfacial polymer layer, consisting of dynamic non-covalent crosslinks, between a semiconducting thin film and a substrate. This enables high interfacial toughness between the layers, suppression of delamination and delocalization of strain. As a result, crack initiation and propagation are notably delayed to much higher strains. Specifically, the crack-onset strain of a high-mobility semiconducting polymer thin film improved from 30% to 110% strain without any noticeable microcracks. Despite the presence of a large mismatch in strain between the plastic semiconducting thin film and elastic substrate after unloading, the tough interface layer helped maintain bonding and exceptional cyclic durability and robustness. Furthermore, we found that our interfacial layer reduces the mismatch of thermal expansion coefficients between the different layers. This approach can improve the crack-onset strain of various semiconducting polymers, conducting polymers and even metal thin films.

Stretchable PPG sensor with light polarization for physical activity-permissible monitoring.

Skin-attachable sensors, which represent the ultimate form of wearable electronic devices that ensure conformal contact with skin, suffer from motion artifact limitations owing to relative changes in position between the sensor and skin during physical activities. In this study, a polarization-selective structure of a skin-conformable photoplethysmographic (PPG) sensor was developed to decrease the amount of scattered light from the epidermis, which is the main cause of motion artifacts. The motion artifacts were suppressed more than 10-fold in comparison with those of rigid sensors. The developed sensor-with two orthogonal polarizers-facilitated successful PPG signal monitoring during wrist angle movements corresponding to high levels of physical activity, enabling continuous monitoring of daily activities, even while exercising for personal health care.

Standalone real-time health monitoring patch based on a stretchable organic optoelectronic system.

Skin-like health care patches (SHPs) are next-generation health care gadgets that will enable seamless monitoring of biological signals in daily life. Skin-conformable sensors and a stretchable display are critical for the development of standalone SHPs that provide real-time information while alleviating privacy concerns related to wireless data transmission. However, the production of stretchable wearable displays with sufficient pixels to display this information remains challenging. Here, we report a standalone organic SHP that provides real-time heart rate information. The 15-µm-thick SHP comprises a stretchable organic light-emitting diode display and stretchable organic photoplethysmography (PPG) heart rate sensor on all-elastomer substrate and operates stably under 30% strain using a combination of stress relief layers and deformable micro-cracked interconnects that reduce the mechanical stress on the active optoelectronic components. This approach provides a rational strategy for high-resolution stretchable displays, enabling the production of ideal platforms for next-generation wearable health care electronics.

Multifunctional materials for implantable and wearable photonic healthcare devices.

Numerous light-based diagnostic and therapeutic devices are routinely used in the clinic. These devices have a familiar look as items plugged in the wall or placed at patients' bedsides, but recently, many new ideas have been proposed for the realization of implantable or wearable functional devices. Many advances are being fuelled by the development of multifunctional materials for photonic healthcare devices. However, the finite depth of light penetration in the body is still a serious constraint for their clinical applications. In this Review, we discuss the basic concepts and some examples of state-of-the-art implantable and wearable photonic healthcare devices for diagnostic and therapeutic applications. First, we describe emerging multifunctional materials critical to the advent of next-generation implantable and wearable photonic healthcare devices and discuss the path for their clinical translation. Then, we examine implantable photonic healthcare devices in terms of their properties and diagnostic and therapeutic functions. We next describe exemplary cases of noninvasive, wearable photonic healthcare devices across different anatomical applications. Finally, we discuss the future research directions for the field, in particular regarding mobile healthcare and personalized medicine.

Author Correction: Surface plasmon enhanced Organic color image sensor with Ag nanoparticles coated with silicon oxynitride.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Surface plasmon enhanced Organic color image sensor with Ag nanoparticles coated with silicon oxynitride.

As organic photodetectors with less than 1 µm pixel size are in demand, a new way of enhancing the sensitivity of the photodetectors is required to compensate for its degradation due to the reduction in pixel size. Here, we used Ag nanoparticles coated with SiOxNy as a light-absorbing layer to realize the scale-down of the pixel size without the loss of sensitivity. The surface plasmon resonance appeared at the interface between Ag nanoparticles and SiOxNy. The plasmon resonance endowed the organic photodetector with boosted photon absorption and external quantum efficiency. As the Ag nanoparticles with SiOxNy are easily deposited on ITO/SiO2, it can be adapted into various organic color image sensors. The plasmon-supported organic photodetector is a promising solution for realizing color image sensors with high resolution below 1 µm.

Green-light-selective organic photodiodes for full-color imaging.

In this work, organic photodiodes (OPDs) based on two newly synthesized p-type dipolar small molecules are reported for application to green-light-selective OPDs. In order to reduce the blue-color absorption induced by the use of C60 as the n-type material in a bulk heterojunction (BHJ), the electron donor:electron acceptor composition ratio is tuned in the BHJ. With this light manipulation approach, the blue-wavelength external quantum efficiency (EQE) is minimized to 18% after reducing the C60 concentration in the center part of the BHJ. The two p-type molecules get a cyanine-like character with intense and sharp absorption in the green color by adjusting the strength of their donating and accepting parts and by choosing a selenophene unit as a π-linker. When combined to C60, the green-wavelength EQE reaches 70% in a complete device composed of two transparent electrodes. Finally, the optical simulation shows the good color-balance performance of hybrid full-color image sensor without an additional filter by using the developed green OPD as the top-layer in stacked device architecture.

The role of defects in organic image sensors for green photodiode.

Controlling defect states in a buffer layer for organic photo devices is one of the vital factors which have great influence on the device performance. Defect states in silicon oxynitride (SiOxNy) buffer layer for organic photo devices can be controlled by introducing appropriate dopant materials. We performed ab initio simulations to identify the effect on doping SiOxNy with carbon (C), boron (B), and phosphorous (P) atoms. The results unveil that hole defects in the SiOxNy layer diminish with the phosphorous doping. Based on the simulation results, we fabricate the small molecule organic photodetector (OPD) including the phosphorous-doped SiOxNy buffer layer and the active film of blended naphthalene-based donor and C60 acceptor molecules, which shows excellent enhancement in the external quantum efficiency (EQE). The results of our charge-based deep level transient spectroscopy (Q-DLTS) measurements confirmed that the EQE enhancement originates from the decrease of the hole traps induced by the reduced hole defects. The method of controlling the defect states in SiOxNy buffer layers by the doping can be used to improve the performance in various organic photo devices.

Narrow-Band Organic Photodiodes for High-Resolution Imaging.

There are growing opportunities and demands for image sensors that produce higher-resolution images, even in low-light conditions. Increasing the light input areas through 3D architecture within the same pixel size can be an effective solution to address this issue. Organic photodiodes (OPDs) that possess wavelength selectivity can allow for advancements in this regard. Here, we report on novel push-pull D-π-A dyes specially designed for Gaussian-shaped, narrow-band absorption and the high photoelectric conversion. These p-type organic dyes work both as a color filter and as a source of photocurrents with linear and fast light responses, high sensitivity, and excellent stability, when combined with C60 to form bulk heterojunctions (BHJs). The effectiveness of the OPD composed of the active color filter was demonstrated by obtaining a full-color image using a camera that contained an organic/Si hybrid complementary metal-oxide-semiconductor (CMOS) color image sensor.

Organic-on-silicon complementary metal-oxide-semiconductor colour image sensors.

Complementary metal-oxide-semiconductor (CMOS) colour image sensors are representative examples of light-detection devices. To achieve extremely high resolutions, the pixel sizes of the CMOS image sensors must be reduced to less than a micron, which in turn significantly limits the number of photons that can be captured by each pixel using silicon (Si)-based technology (i.e., this reduction in pixel size results in a loss of sensitivity). Here, we demonstrate a novel and efficient method of increasing the sensitivity and resolution of the CMOS image sensors by superposing an organic photodiode (OPD) onto a CMOS circuit with Si photodiodes, which consequently doubles the light-input surface area of each pixel. To realise this concept, we developed organic semiconductor materials with absorption properties selective to green light and successfully fabricated highly efficient green-light-sensitive OPDs without colour filters. We found that such a top light-receiving OPD, which is selective to specific green wavelengths, demonstrates great potential when combined with a newly designed Si-based CMOS circuit containing only blue and red colour filters. To demonstrate the effectiveness of this state-of-the-art hybrid colour image sensor, we acquired a real full-colour image using a camera that contained the organic-on-Si hybrid CMOS colour image sensor.

Enhancement of diffuse reflectance using air tunnel structure.

Submicrometer air gap structure has formed on diffuse reflection structure to improve light reflectance. Covering polymer or liquid on a diffuse reflector to make optical components induces the severe decrease of the total reflectance, since the diffuse reflected angle of some light rays is larger than the critical angle and the rays travel to the medium until meeting a proper small incident angle. The reflectance drops to 68% of the original value with just a polymer coating on the diffuse reflector. The formation of an air tunnel structure between the polymer layer and the diffuse reflector makes a symmetrical reflective index matching state and recovers 95% of the original reflectance. Due to the simple fabrication process and the chemical stability, the structure can be applied to various optical components and reflective display devices.

Bright color optical switching device by polymer network liquid crystal with a specular reflector.

The color optical switching device by polymer network liquid crystal (PNLC) with color filter on a specular reflector shows excellent performance; white reflectance of 22%, color gamut of 32%, and contrast ratio up to 50:1 in reflective mode measurement. The view-angle dependence of the reflectance can be adjusted by changing the PNLC thickness. The color chromaticity shown by the device is close to the limit value of color filters, and its value nearly remains with respect to the operating voltage. These optical properties of the device can be explained from the prediction based on multiple interactions between the light and the droplets of liquid crystal. The high reflectance, vivid color image, and moderate responds time allow the PNLC device to drive good color moving image. It can widely extend the applications of the reflective device.

Measurement of the polarization of high-order harmonics from aligned N(2) molecules by spatial interferometry.

The polarization of high-harmonics from aligned N(2) molecules was measured by observing the visibility of spatial interference between two high-harmonics generated separately. The minimum visibility was observed at an angle of 60 degrees between the polarization of the harmonic generation laser field and the molecular orientation. In this case, the angular shift of harmonic polarization is 15 degrees from the molecular orientation. Our measurement of the visibility variation matches the theoretical prediction based on the harmonic field calculation for aligned N(2) molecules.