Integrating recognition and reasoning for tracking and querying in smart environments

Abstract

This dissertation addresses the problem of unobtrusive identification and tracking of occupants in smart environments and answering queries about their whereabouts. Our main contribution is a model for a smart environment based upon a novel state transition system in which each state records the set of occupants who are present in the various zones of the environment. An event abstracts a biometric recognition step, and the transition function abstracts the reasoning necessary to effect state transitions. Since biometric recognition is inexact, state information is inherently probabilistic in nature. Our model supports recognition based upon multiple modalities such as face, gait, and voice in a uniform manner, and also supports fusion of two or more modalities. Another contribution of our research lies in showing how the identity of a person is determined by spatio-temporal reasoning over a track of events rather than based upon a single event. Thus, the integration of recognition and reasoning helps overcome some of the inherent limitations of biometric recognition. In this dissertation, we also provide a quantification of the performance of a smart environment by refining the standard information-theoretic concepts of precision and recall to suit our context. We explore two broad approaches to performance characterization: query-independent and query-dependent precision and recall metrics. The queries of interest are probabilistic and spatio-temporal in nature, e.g., ‘What is the probability that A and B met in the high security zone of the building in the afternoon?’. Thus, we postulate that the 3 R’s of a smart environment are recognition, reasoning, and retrieval. We have developed an experimental system for a smart environment in order to validate our concepts, and we provide experimental results in this dissertation.