Segmentation and classification of moving video objects

Despite very optimistic predictions in the early days of Artificial Intelligence research, a computer vision system that interprets image sequences acquired from arbitrary real-world scenes still remains out of reach. Nevertheless, there has been great progress in the field since then and a number of applications emerged within different areas. Of particular interest for several applications are capabilities for object segmentation and object recognition. Algorithms from the former category support the segmentation of the observed world into semantic entities, thus allow a transition from signal processing towards an object-oriented view. Object recognition approaches allow the classification of objects into categories and enable for conceptual representations of still images or videos. The goal of this chapter is the development of a classification system for objects that appear in videos. This information can be used to index or categorize videos and it thus supports object-based video retrieval. In order to keep the subject manageable, the system is embedded into a set of constraints: The segmentation module relies on motion information, thus it can only detect moving objects. Furthermore, the classification module only considers the two-dimensional shape of the segmented objects. Therefore, just a coarse classification of the objects into generic classes (e.g., cars, people) is possible.