“Magic Mirror in my Hand, what is the Sentiment in the Lens?”: an Action Unit based Approach for Mining Sentiments from Multimedia Contents

Emotions are an increasingly important factor in Human-Computer Interaction. So, extracting emotions from multimedia contents is becoming one of the most challenging research topics in Computer Science. Facial expressions, posture, gestures, speech, emotive changes of physical parameter (e.g. body temperature, blush and changes in the tone of the voice) can reflect changes in the user's emotional state. All this kind of parameters can be detected and interpreted by a computer leading to the so-called “affective computing”. Through affective computing, client's posture, gestures, and facial expressions could be used, along with words, for a more accurate evaluation of their psychological state. In this paper an approach for the extraction of emotions from images will be introduced. The proposed framework involves the adoption of action units’ extraction from facial expression according to the Ekman theory. The proposed approach has been tested on standard datasets and the results are interesting and promising.