Hemispherical Retina Emulated by Plasmonic Optoelectronic Memristors with All-Optical Modulation for Neuromorphic Stereo Vision.

Binocular stereo vision relies on imaging disparity between two hemispherical retinas, which is essential to acquire image information in three dimensional environment. Therefore, retinomorphic electronics with structural and functional similarities to biological eyes are always highly desired to develop stereo vision perception system. In this work, a hemispherical optoelectronic memristor array based on Ag-TiO2 nanoclusters/sodium alginate film is developed to realize binocular stereo vision. All-optical modulation induced by plasmonic thermal effect and optical excitation in Ag-TiO2 nanoclusters is exploited to realize in-pixel image sensing and storage. Wide field of view (FOV) and spatial angle detection are experimentally demonstrated owing to the device arrangement and incident-angle-dependent characteristics in hemispherical geometry. Furthermore, depth perception and motion detection based on binocular disparity have been realized by constructing two retinomorphic memristive arrays. The results demonstrated in this work provide a promising strategy to develop all-optically controlled memristor and promote the future development of binocular vision system with in-sensor architecture.

Artifact reduction in photoacoustic images by generating virtual dense array sensor from hemispheric sparse array sensor using deep learning.

PURPOSE: Vascular distribution is important information for diagnosing diseases and supporting surgery. Photoacoustic imaging is a technology that can image blood vessels noninvasively and with high resolution. In photoacoustic imaging, a hemispherical array sensor is especially suitable for measuring blood vessels running in various directions. However, as a hemispherical array sensor, a sparse array sensor is often used due to technical and cost issues, which causes artifacts in photoacoustic images. Therefore, in this study, we reduce these artifacts using deep learning technology to generate signals of virtual dense array sensors. METHODS: Generating 2D virtual array sensor signals using a 3D convolutional neural network (CNN) requires huge computational costs and is impractical. Therefore, we installed virtual sensors between the real sensors along the spiral pattern in three different directions and used a 2D CNN to generate signals of the virtual sensors in each direction. Then we reconstructed a photoacoustic image using the signals from both the real sensors and the virtual sensors. RESULTS: We evaluated the proposed method using simulation data and human palm measurement data. We found that these artifacts were significantly reduced in the images reconstructed using the proposed method, while the artifacts were strong in the images obtained only from the real sensor signals. CONCLUSION: Using the proposed method, we were able to significantly reduce artifacts, and as a result, it became possible to recognize deep blood vessels. In addition, the processing time of the proposed method was sufficiently applicable to clinical measurement.