Predicting Personal Traits from Facial Images Using Convolutional Neural Networks Augmented with Facial Landmark Information - Lewenberg, Y.; Bachrach, I.; Shankar, S.; Criminisi, Antonio (AAAI, 2016)
We consider the task of predicting various traits of a person given an image of their face. We aim to estimate traits such as gender, ethnicity and age, as well as more subjective traits as the emotion a person expresses or whether they are humorous or attractive. Due to the recent surge of research on Deep Convolutional Neural Networks (CNNs), we begin by using a CNN architecture, and corroborate that CNNs are promising for facial attribute prediction. To further improve performance, we propose a novel approach that incorporates facial landmark information for input images as an additional channel, helping the CNN learn face-specific features so that the landmarks across various training images hold correspondence. We empirically analyze the performance of our proposed method, showing consistent improvement over the baselines across traits. We demonstrate our system on a sizeable Face Attributes Dataset (FAD), comprising of roughly 200,000 labels, for 10 most sought-after traits, for over 10,000 facial images.