Accession Number:



Ensemble Learning Method for Hidden Markov Models

Descriptive Note:

Technical Report

Corporate Author:

University of Louisville Louisville United States

Personal Author(s):

Report Date:


Pagination or Media Count:



This dissertation introduces an ensemble learning method for temporal data that uses a mixture of Hidden Markov Model HMM classfiers. We hypothesize that the data is generated by K models, each of which reflects a particular trend in the data. Model identfication could be achieved through clustering in the feature space or in the parameters space. However, this approach is inappropriate in the context of sequential data. The proposed approach is based on clustering in the log-likelihood space, and has two main steps. First, one HMM is fit to each of the N individual sequences. For each fitted model, we evaluate the log-likelihood of each sequence. This will result in an N-by-N log-likelihood distance matrix that will be partitioned into K groups using a relational clustering algorithm. In the second step, we pool the sequences belonging to the same cluster into K groups. Then, we learn the parameters of one HMM per group. We propose using and optimizing various training approaches for the different K groups depending on their size and homogeneity. In particular, we investigate the maximum likelihood ML, the minimum classification error MCE based discriminative, and the Variational Bayesian VB training approaches. Finally, to test a new sequence, its likelihood is computed in all the models and a final confidence value is assigned by combining the multiple models outputs using a decision level fusion method such as an artficial neural network or a hierarchical mixture of experts. Our approach was evaluated on two real-world applications 1 identification of Cardio-Pulmonary Resuscitation CPR scenes in video simulating medical crises and 2 landmine detection using Ground Penetrating Radar GPR. Results on both applications show that the proposed method can identify meaningful and coherent HMM mixture components that describe different properties of the data. Each HMM mixture component models a group of data that share common attributes.

Subject Categories:

  • Statistics and Probability

Distribution Statement: