Recognizing Fine Facial Micro-Expressions Using Two-Dimensional Landmark Feature

Emotion recognition based on facial expressions is very important for interaction between human and artificial intelligence (AI) system such as social robots. On the other hand, it is much harder to recognize subtle facial expressions or facial micro-expressions than facial expressions rich in emotional expression in a real environment. In this paper, we propose a two-dimensional (2D) landmark feature for effectively recognizing facial micro-expression. The proposed 2D landmark feature is obtained by converting existing coordinate-based landmark information into 2D image information, and has an advantage of having a unique feature according to emotions regardless of the intensity of facial expression. Thus, we can achieve effective emotion recognition by learning the proposed 2D landmark feature information on a convolutional neural network (CNN) and a long-term term memory (LSTM)-based network. Experimental results show that the proposed method provides more than 77% classification performance for fine facial expression images even when learning with general facial expression images of CK+ dataset.