Accession Number : ADA448064


Title :   Automatic Performance Evaluation for Video Summarization


Descriptive Note : Technical rept.


Corporate Author : MARYLAND UNIV COLLEGE PARK INST FOR ADVANCED COMPUTER STUDIES


Personal Author(s) : Huang, Mei ; Mahajan, Ayesh B ; DeMenthon, Daniel F


Full Text : https://apps.dtic.mil/dtic/tr/fulltext/u2/a448064.pdf


Report Date : Jul 2004


Pagination or Media Count : 33


Abstract : This paper describes a system for automated performance evaluation of video summarization algorithms. We call it SUPERSIEV (System for Unsupervised Performance Evaluation of Ranked Summarization in Extended Videos). It is primarily designed for evaluating video summarization algorithms that perform frame ranking. The task of summarization is viewed as a kind of database retrieval, and we adopt some of the concepts developed for performance evaluation of retrieval in database systems. First, ground truth summaries are gathered in a user study from many assessors and for several video sequences. For each video sequence, these summaries are combined to generate a single reference file that represents the majority of assessors opinions. Then the system determines the best target reference frame for each frame of the whole video sequence and computes matching scores to form a lookup table that rates each frame. Given a summary from a candidate summarization algorithm, the system can then evaluate this summary from different aspects by computing recall, cumulated average precision, redundancy rate and average closeness. With this evaluation system, we can not only grade the quality of a video summary, but also (1) compare different automatic summarization algorithms and (2) make stepwise improvements on algorithms, without the need for new user feedback.


Descriptors :   *ALGORITHMS , *INFORMATION RETRIEVAL , *SURFACE TRUTH , *RANKING , *VIDEO FRAMES , DATA BASES , AUTOMATION , USER NEEDS , FILM READERS , SCORING , QUALITY ASSURANCE , TABLES(DATA) , HIGH VELOCITY , SEQUENTIAL ANALYSIS , PRECISION


Subject Categories : Information Science
      Computer Programming and Software
      Computer Systems


Distribution Statement : APPROVED FOR PUBLIC RELEASE