Results from Evaluation of Three Commercial Off the shelf Face Recognition Systems on Chokepoint Dataset
Canada Border Services Agency Ottawa ON Canada
Pagination or Media Count:
This report is a supplement to Division Report 2014-27 TR Evaluation methodology for face recognition technology in video surveillance applications by E. Granger and D. Gorodnichy 1. It presents complete evaluation results of three Commercial Off-The-Shelf COTS Face Recognition FR systems Cognitec, PittPatt and Neurotechnology - obtained on the Chokepoint public data-set using the multi-level evaluation methodology introduced in the previous report. Four levels of details are examined according to the methodology for each target person from the Checkpoint data-set Level 0 or score-based analysis illustrates the probability distribution of the genuine scores against that of the impostors at different face resolutions, which visually illustrates the discrimination power of the system for each target individual. Level 1 or transaction-based analysis provides the averaged description of the system in terms of false positive and false negative rates aggregated over all transactions, expressed using Receiver Operative Curves ROC, Detection Error Trade-off DET, and Precision-Recall Operating Characteristic PROC curves. Level 2 or subject-based analysis describes the performance of the system using the-so-called Doddingtons Zoo categorization of individuals, which detects whether an individual belongs to an easier or a harder classes of people that the system is able to recognize 2, 3. Finally, Level 3 or temporal analysis allows one to examine the overall discrimination power of the system by accumulating the positive predictions while tracking a person over time and computing the recognition confidence based thereon. Two-page Report Cards summarizing the performance of the system for each target individual are published, thus providing an exhaustive report of the systems performance on a variety of different target subjects.