Learning to recognize video-based spatiotemporal events

A key research issue in activity recognition in real-world applications, such as in intelligent transportation systems (ITS), is to automatically learn robust models of activities that require minimal human training. In this paper, we contribute a novel approach for learning sequenced spatiotemporal activities in outdoor traffic intersections. Concretely, by representing the activities as sequences of actions, we contribute a semisupervised learning algorithm that learns activities as complete stochastic context-free grammars (SCFGs), namely, the grammar structure and the parameters. Our approach has been implemented and tested on real-world scenes, and we present experimental results of the grammar learning and activity recognition applied to datacollection and traffic monitoring applications using video data.