Toward High-quality Face-Mask Occluded Restoration

ABSTRACT

Face-mask occluded restoration aims at restoring the masked region of a human face, which has attracted increasing attention in the context of the COVID-19 pandemic. One major challenge of this task is the large visual variance of masks in the real world. To solve it we first construct a large-scale Face-mask Occluded Restoration (FMOR) dataset, which contains 5,500 unmasked images and 5,500 face-mask occluded images with various illuminations, and involves 1,100 subjects of different races, face orientations, and mask types. Moreover, we propose a Face-Mask Occluded Detection and Restoration (FMODR) framework, which can detect face-mask regions with large visual variations and restore them to realistic human faces. In particular, our FMODR contains a self-adaptive contextual attention module specifically designed for this task, which is able to exploit the contextual information and correlations of adjacent pixels for achieving high realism of the restored faces, which are however often neglected in existing contextual attention models. Our framework achieves state-of-the-art results of face restoration on three datasets, including CelebA, AR, and our FMOR datasets. Moreover, experimental results on AR and FMOR datasets demonstrate that our framework can significantly improve masked face recognition and verification performance. © 2023 Association for Computing Machinery.