Detection and Interpretation of Low-Level and High-Level Surprising and Important Events in Large-Scale Data Streams
Technical Report,17 Sep 2012,16 Mar 2016
UNIVERSITY OF SOUTHERN CALIFORNIA LOS ANGELES LOS ANGELES United States
Pagination or Media Count:
This project explored how to mathematically formalize the computations of surprise and relevance of events in large data streams, including video, audio and text. We have developed new mathematical theories to define surprise in terms of how new data observations may or not affect an observers set of beliefs. This is computed in terms of the Kullback-Leibler divergence between posterior and prior beliefs of the observer, and quantified in a new unit of wows. Likewise, we have developed a new general theory of relevance that quantifies how new data observations may or not affect an observers beliefs about how sheheit will achieve its goals. Data observations which suggest that some previously possible solutions to a problem are now invalid will be measured as more relevant, in a new unit of rels. Both theories have been extensively tested using large video 3000 hours and text twitter feeds datasets.