Cognitive name-face association through context-aware Graph Neural Network

The extraction of valuable insights from unstructured content has attracted much attention in the last decades. Main results lie in the area of text mining, while the understanding of multimedia contents, thanks to the improvements in computer vision, mainly relies on adopting emerging deep learning models. About image understanding, people’s name association in images is still an open issue. The approaches at the state of the art mainly use facial features and find the corresponding names by extracting the most recurring entities in the attached captions. These methods are experimented for celebrities and often fail when few labeled samples are available or there are particular poses. The proposed solution tries to improve the name–face association in such cases by defining a cognitive layer for a deep learning architecture embedding the surrounding context of the entities in the caption or the image. The method mainly focuses on name–face association as enabling technology for people recognition in open-source intelligence frameworks that mostly investigate not popular (or unknown) people. Given a face, the proposed system predicts the most likely corresponding name leveraging image features, caption, and context. The learning model embeds the knowledge graph of the context elicited in the open sources through a graph neural network. The experimentation highlights that the context improves the results of the name–face association, especially when other methods fail.