Convolutional neural network and ensemble machine learning model for optimizing performance of emotion recognition in wild

Emotions play a pivotal role in our everyday interactions, serving as a crucial indicator of a speaker’s influence. Among various means of expression, facial emotions hold a special prominence compared to hand gestures, body movements, and more. The expressions on the face serve as a primary canvas for conveying these emotions. Numerous studies have introduced approaches and models for recognizing emotions. However, it is important to note that many of these approaches have been predominantly tested on datasets gathered in controlled environments. In contrast, real-world environments are dynamic and unpredictable, introducing numerous challenges to emotion recognition. The current study pursues a two-way approach to emotion recognition. Firstly, the research introduces a novel face mask dataset, which is designed for emotion recognition in a real-time setting. This dataset is utilized in conjunction with a convolutional neural network (CNN), and multiple image processing techniques are applied to the manually labeled face mask dataset. The second phase explores emotion recognition through textual data, employing a range of features, including hand-crafted, Word Embedding, Fast Text embedding, and Transformer models. The study compares the models’ performance using original features versus convoluted features. The study aims to provide valuable insights into emotion recognition through both textual and face-masked data, contributing to our understanding of this important field.