Holistically Constrained Local Model: Going Beyond Frontal Poses for Facial Landmark Detection
University of Southern California, Institute for Robotics and Intelligent Systems Los Angeles United States
Pagination or Media Count:
Facial landmark detection has received much attention in recent years, with two detection paradigms emerging local approaches, where each facial landmark is modeled individually and with the help of a shape model and holistic approaches, where the whole face appearance is jointly modeled for improved robustness. These approaches have shown great performance for facial landmark detection even under in-the-wild conditions of varying illumination, occlusion and image quality. However, their accuracy and robustness are very often reduced for profile faces where face alignment is more challenging e.g., no more facial symmetry, less defined features and more variable background. In this paper, we present a new model, named Holistically Constrained Local Model HCLM, which unifies local and holistic facial landmark detection by integrating head pose estimation, sparse-holistic landmark detection and dense-local landmark detection. We evaluate our new model on two publicly available datasets, 300-W and AFLW, as well as a newly introduced dataset, IJB-FL which includes a larger proportion of profile face poses. Our HCLM model shows state-of-the-art performance, especially with extreme head poses.