Graph Network based Approaches for Multi-modal Movie Recommendation System

ABSTRACT

The three times increase of SonyLiv viewers during the Tokyo Olympic, the 10% hike of YouTube users during the isolation era of covid-pandemic, and the 19% growth in Netflix user count due to the fastest growth of OTT, etc. have made the digital platform's mode all-time active and specific. The hourly increase of users' interactions and the e-commerce platform's desire of letting users engage on their sites are pushing researchers to shape the virtual digital web as user specific and revenue-oriented. This paper develops a deep learning-based approach for building a movie recommendation system with three main aspects (a) using a knowledge graph to embed text and meta information of movies, (b) using multi-modal information of movies like audio, visual frames, text summary, meta data information to generate movie/user representations without directly using rating information;this multi-modal representation can help in coping up with cold-start problem of recommendation system (c) a graph attention network based approach for developing regression system. For meta encoding, we have built knowledge graph from the meta information of the movies directly. For movie-summary embedding, we extracted nouns, verbs, and object to build a knowledge graph with head-relation-tail relationships. A deep neural network, as well as Graph attention networks, are utilized for measuring performance in terms of RMSE score. The proposed system is tested on an extended MovieLens-100K data-set having multi-modal information. Experimental results establish that only rating-based embeddings in the current setup outperform the state-of-the-art techniques but usage of multi-modal information in embedding generation performs better than its single-modal counterparts. 1. © 2022 IEEE.