Action Bank: A High Level Representation of Activity in Video (Author's Manuscript)
University at Buffalo - SUNY Buffalo United States
Pagination or Media Count:
Activity recognition in video is dominated by low- and mid-level features, and while demonstrably capable, by nature, these features carry little semantic meaning. Inspired by the recent object bank approach to image representation, we present Action Bank, a new high-level representation of video. Action bank is comprised of many individual action detectors sampled broadly in semantic space as well as viewpoint space. Our representation is constructed to be semantically rich and even when paired with simple linear SVM classifiers is capable of highly discriminative performance. We have tested action bank on four major activity recognition benchmarks. In all cases, our performance is better than the state of the art, namely 98.2 on KTH better by 3.3 , 95.0 on UCF Sports better by 3.7 , 57.9 on UCF50 baseline is 47.9 , and 26.9 on HMDB51 baseline is 23.2 . Furthermore, when we analyze the classifiers, we find strong transfer of semantics from the constituent action detectors to the bank classifier.