ENHANCING HUMAN-CENTRIC INTERACTION: A DEEP LEARNING APPROACH FOR ROBUST FACIAL EXPRESSION RECOGNITION AND INTENSITY ESTIMATION

Abstract

This paper presents a deep learning framework for real-time facial expression recognition and expression in-tensity estimation using spontaneous video data. Our system effectively tackles the most important real-world challenges like subtle variations of expressions, intersubject variability, and variations of head pose or illumination using a lightweight convolutional neural network architecture. We achieve 68.35% classification accuracy on spontaneous happy/neutral expressions with a supporting dataset of 1,442 frames annotated on a 7-point intensity scale (Cohen’s = 0.81) and show that the network can process 720p video on embedded hardware at 28 fps. The key contributions are as follows: (1) an optimized CNN model for joint classification and regression; (2) a novel postprocess-ing pipeline for context-aware intensity smoothing; and (3) an ethically collected dataset emphasizing spontaneous dynamics. Our results outperform existing baselines of spontaneous FER by 3.35%, underlining the potential of this lightweight approach in healthcare, HCI, and automotive safety.