Recognizing activities in real-world videos is a challenging AI problem. We present a novel combination of standard activity classification, object recognition, and text mining to learn effective activity recognizers without ever explicitly labeling training videos. We cluster verbs used to describe videos to automatically discover classes of activities and produce a labeled training set. This labeled data is then used to train an activity classifier based on spatio-temporal features. Next, text mining is employed to learn the correlations between these verbs and related objects. This knowledge is then used together with the outputs of an off-the-shelf object recognizer and the trained activity classifier to produce an improved activity recognizer. Experiments on a corpus of YouTube videos demonstrate the effectiveness of the overall approach.

PDF

In Proceedings of the 20th European Conference on Artificial Intelligence (ECAI-2012), pp. 600--605, August 2012.

@inproceedings{motwani:ecai12, title={Improving Video Activity Recognition using Object Recognition and Text Mining }, author={Tanvi S. Motwani and Raymond J. Mooney}, booktitle={Proceedings of the 20th European Conference on Artificial Intelligence (ECAI-2012)}, month={August}, pages={600--605}, url="http://www.cs.utexas.edu/users/ai-lab?motwani:ecai12", year={2012} }

Slides (PPT)