A robust self-supervised image hashing method for content identification with forensic detection of content-preserving manipulations.

Image content identification systems have many applications in industry and academia. In particular, a hash-based content identification system uses a robust image hashing function that computes a short binary identifier summarizing the perceptual content in a picture and is invariant against a set of expected manipulations while being capable of differentiating between different pictures. A common approach to designing these algorithms is crafting a processing pipeline by hand. Unfortunately, once the context changes, the researcher may need to define a new function to adapt. A deep hashing approach exploits the feature learning capabilities in deep networks to generate a feature vector that summarizes the perceptual content in the image, achieving outstanding performance for the image retrieval task, which requires measuring semantic and perceptual similarity between items. However, its application to robust content identification systems is an open area of opportunity. Also, image hashing functions are valuable tools for image authentication. However, to our knowledge, its application to content-preserving manipulation detection for image forensics tasks is still an open research area. In this work, we propose a deep hashing method exploiting the metric learning capabilities in contrastive self-supervised learning with a new modular loss function for robust image hashing. Moreover, we propose a novel approach for content-preserving manipulation detection for image forensics through a sensitivity component in our loss function. We validate our method through extensive experimentation in different data sets and configurations, validating the generalization properties in our work.

Robust image hashing for content identification through contrastive self-supervised learning.

Content identification systems are an essential technology for many applications. These systems identify query multimedia items using a database of known identities. A hash-based system uses a perceptual hashing function that generates a hash value invariant against a set of expected manipulations in an image, later compared to perform identification. Usually, this set of manipulations is well-known, and the researcher creates the perceptual hashing function that best adapts to the set. However, a new manipulation may break the hashing function, requiring to create a new one, which may be costly and time-consuming. Therefore, we propose to let the hashing function learn an invariant feature space automatically. For this, we exploit the recent advances in self-supervised learning, where a model uses unlabeled data to generate a feature representation by solving a metric learning-based pretext task that enforces the robust image hashing properties for content identification systems. To achieve model transferability on unseen data, our pretext task enforces the feature vector invariance against the manipulation set, and through random sampling on the unlabeled training set, we present the model a wide variety of perceptual information to work on. As exhaustive experimentation shows, this method achieves excellent robustness against a comprehensive set of manipulations, even difficult ones such as horizontal flip and rotation, with excellent identification performance. Also, the trained model is highly discriminative against the presence of near-duplicate images. Furthermore, this method does not need re-training or fine-tuning on a new dataset to achieve the observed performance, indicating an excellent generalization capacity.