Semantic representation learning for a mask-modulated lensless camera by contrastive cross-modal transferring.

Lensless computational imaging, a technique that combines optical-modulated measurements with task-specific algorithms, has recently benefited from the application of artificial neural networks. Conventionally, lensless imaging techniques rely on prior knowledge to deal with the ill-posed nature of unstructured measurements, which requires costly supervised approaches. To address this issue, we present a self-supervised learning method that learns semantic representations for the modulated scenes from implicitly provided priors. A contrastive loss function is designed for training the target extractor (measurements) from a source extractor (structured natural scenes) to transfer cross-modal priors in the latent space. The effectiveness of the new extractor was validated by classifying the mask-modulated scenes on unseen datasets and showed the comparable accuracy to the source modality (contrastive language-image pre-trained [CLIP] network). The proposed multimodal representation learning method has the advantages of avoiding costly data annotation, being more adaptive to unseen data, and usability in a variety of downstream vision tasks with unconventional imaging settings.

Lensless facial recognition with encrypted optics and a neural network computation.

Face recognition plays an essential role for the biometric authentication. Conventional lens-based imagery keeps the spatial fidelity with respect to the object, thus, leading to the privacy concerns. Based on the point spread function engineering, we employed a coded mask as the encryption scheme, which allows a readily noninterpretable representation on the sensor. A deep neural network computation was used to extract the features and further conduct the identification. The advantage of this data-driven approach lies in that it is neither necessary to correct the lens aberration nor revealing any facial conformity amid the image formation chain. To validate the proposed framework, we generated a dataset with practical photographing and data augmentation by a set of experimental parameters. The system has the capability to adapt a wide depth of field (DoF) (60-cm hyperfocal distance) and pose variation (0 to 45 deg). The 100% recognition accuracy on real-time measurement was achieved without the necessity of any physics priors, such as the encryption scheme.