Indexing Key Positions

This project was my DEA (French Masters) thesis work advised by Radu Horaud.

In this work, given two or more video sequences containing similar human activities (running, jumping, etc.) we devise a method which extracts spatio-temporal signatures associated with these activities, compares these signatures, and aligns key positions of different videos. An example application domain is sport events. Currently, individual performances such as jumps and vaults (athletics, gymnastics, etc.) are difficult to compare quantitatively, from one athlete to another, because the only available data is video sequences. In this work we show the possibility of, for e.g., comparing trajectories associated with two high jumps or pole vaults performed by two different athletes at two different events (e.g. different Olympic games) and to automatically index key positions in the second video given a set of key positions from the first video.

Approach and Some Results
Our main focus is on athletic video sequences, for e.g. high jump, pole vault, etc. In the figure 1, we show some sample frames from two such video sequences. Initial frame for both video sequences have been syncronized in time (i.e correspond to same physical state of the activity). Subsequent frames are equally spaced. As evident from the figure, the timings of the jump are quite different. Also note that viewpoint is quite different.

Figure 2 shows the result after indexing key positions.

Figure 1. Input Image sequences from two high jumps. Starting point is syncronized and corresponding frames are equally spaced in time.

Figure 2. Key position indexing results for image sequences in figure 1. Note that corresponding frames now correspond to same physical state of the activity.

How we do it? Briefly, using Motion Panorama system, we calibrate cameras and extract the camera motion parameters i.e. pan, tilt angles and focal length at each time instant. Athlete's center of gravity (CG) or torso position is tracked automatically using CONDENSATION based framework. These CG positions are then viewed on an adaptive manifold mosaic providing a space-time curve of athlete's trajectory. Use of adaptive manifold for curve generation has many advantages detailed in corresponding publication. The figure below shows the two such curves (both before registration and after registration) for athletes doing high jump.

Figure 3. Figure to the left, shows two curves before registration. As can be clearly seen, the time of jump is quite different. On Right is the results after registration. Note that how the various features of two curves are now registered with respect to each other.

Corresponding Publication

Indexing Key Positions between Multiple Videos Navneet Dalal and Radu Horaud MOTION'02 - In Proceedings of the IEEE Workshop on Motion and Video Computing, Orlando, Florida, USA, December 2002.