A Comparison of HMMs and Dynamic Bayesian Networks for Recognizing Office Activities

We present a comparative analysis of a layered architecture of Hidden Markov Models (HMMs) and dynamic Bayesian networks (DBNs) for identifying human activities from multimodal sensor information. We use the two representations to diagnose users’ activities in S-SEER, a multimodal system for recognizing office activity from real-time streams of evidence from video, audio, and computer (keyboard and mouse) interactions. As the computation required for sensing and processing perceptual information can impose significant burdens on personal computers, the system is designed to perform selective perception using expected-value-of-information (EVI) to limit sensing and analysis. We discuss the relative performance of HMMs and DBNs in the context of diagnosis and EVI computation.