Event Detection for Streaming Analytics: An Intelligent Mathematical Paradigm

Major Goals: The objective of the proposed research is twofold. First, from the scientific or mathematical aspect, we propose to create an innovative, reliable and scalable event detection prototype with theoretical guarantees for streaming data. The proposed work begins from conceptually differentiating novelties from anomalies, then proceeds to substantially extending our newly-developed streaming data mining system to real-time event detection, and further creating advanced detection algorithms for more pressing yet understudied challenges in mining streaming data. To this end, we developed the following key techniques. 1). An adaptive online kernel density estimation based algorithm to accurately pinpoint isolated anomalies and cohesive novel patterns from unlabeled, concept-drifting data streams with noise; 2). A dynamically evolving recurrent neural network to reveal suspicious rare events either as a semi-supervised model in a finite label latency context or as a supervised model in an infinitely delayed label scenario; 3). An online margin-based learning method to effectively handle evolving feature spaces while performing scalable interpretable event detection over noisy streaming data without expert labels; and 4). A multi-task learning framework to collaboratively conduct reliable and stable event detection when facing multi-source asynchronous raw data streams. All these advances are embodied in an intelligent mathematical paradigm that offers big data practitioners an open-source toolbox of qualitatively different approaches. The ability to learn, distinguish and characterize anomalous and novel signals in (multiple) streaming data would enhance the effectiveness of online risk and threat assessment, prevention, and neutralization, enabling the DoD to renovate a considerable knowledge-centric real-time flow of information to assist in critical strategic decision making.