MSEE: Stochastic Cognitive Linguistic Behavior Models for Semantic Sensing
Final rept. 13 Sep 2011-30 Jul 2013
STEVENS INST OF TECH HOBOKEN NJ
Pagination or Media Count:
This report summarizes the major findings from our research on a semantic information representation framework SIRF for visual sensing scenarios. First, the concept and architecture of a cognitive linguistic CL based SIRF is introduced. Two levels of information abstraction are proposed within this framework. At the syntactic level, a probabilistic contest free grammar PCFG method is employed for information compression and summarization. At the semantic level, a Bayesian network approach is used to achieve semantic concept inference and reasoning. To facilitate the functions of this SIRF, several conceptual primitive modeling methods are proposed, which include a dynamic structure preserving map DSPM for individual human action recognition, a Gaussian Process Dynamic Model with Social Network Analysis GPDM-SNA for a small human group action recognition, an extended GPDM-SNA method for human object interaction HOI recognition, and a pyramid histogram of gradient pHOG method for human object recognition based on gait images. In addition to these conceptual primitive models, two quantities sensing modality utility assessment methods are introduced. They are essentially feature selection methods, one is based sparse imputation and one is based on 11 graph. Extensive experiments on publicly available datasets have been conducted to assess the effectiveness of the proposed methods, and highly competitive and promising results have been observed.