Visual Analysis of High DOF Articulated Objects with Application to Hand Tracking.

Measurement of human hand and body motion is an important task for applications ranging from athletic performance analysis to advanced user-interfaces. Commercial human motion sensors are invasive, requiring the user to wear gloves or targets. This thesis addresses noninvasive real-time 3D tracking of human motion using sequences of ordinary video images. In contrast to other sensors, video cameras are passive and inobtrusive, and can easily be added to existing work environments. Other computer vision systems have demonstrated real-time tracking of a single rigid object in six degrees-of-freedom DOFs. Articulated objects like the hand present three challenges to existing rigid-body tracking algorithms a large number of DOFs 27 for the hand, nonlinear kinematic constraints, and complex self-occlusion effects. This thesis presents a novel tracking framework for articulated objects that uses explicit kinematic models to overcome these obstacles. Kinematic models play two main roles in this work they provide geometric constraints on image features and predict self-occlusions. A kinematic model for hand tracking gives the 3D positions of the fingers as a function of the hand state, which consists of the pose of the palm and the finger joint angles. Image features for the hand consist of lines and points which are obtained by projecting finger phalanges and tips into the image plane. The kinematic model provides a geometric constraint on the image plane positions of hand features as a function of the hand state. Tracking proceeds by registering the projection of the hand model with measured image features at a high frame rate.