Exploring Video Streams using Slit-Tear Visualizations

Authors

• Anthony Tang
• Saul Greenberg
• Sidney Fels

Short description

Video slicing — a variant of slit scanning in photography — extracts a scan line from a video frame and successively adds that line to a composite image over time. The composite image becomes a time line, where its visual patterns reflect changes in a particular area of the video stream. We extend this idea of video slicing by allowing users to draw marks anywhere on the source video to capture areas of interest. These marks, which we call slit-tears, are used in place of a scan line, and the resulting composite timeline image provides a much richer visualization of the video data. Depending on how tears are placed, they can accentuate motion, small changes, directional movement, and relational patterns.

[Tang et al., 2008]

Suitable Datatypes

This method can be applied to live video as well as captured AVI files where each frame is either computed seperately or completely in advance, together with its successors. The latter is only relevant to non-live video.

Figures

Pattern-level analysis:

This image shows how events can be visualized spatially which allows for a discovery of patterns and for an effective correlation of events. Frames t1 to t5 capture a traffic intersection at particular moments in time according to the three slits, drawn by the observer. The interaction of cars and pedestrians is visualized by setting up a timeline for each pre-defined slit where the movement of the objects is depicted by strokes of different length, based on their orientation to each slit.

Traffic intersection showing the interaction patterns of cars and pedestrians

Important citations

More generally, events of interest can be problematic to see in conventional replay of video when the image quality may be poor, events may be very brief or spatially small, and patterns over time may be hard to detect. Slit-tears are a technique that helps to overcome these difficulties.

[Tang et al., 2008]

The interactivity of slit-tear placement at any time (inclding clearing old slit-tears) means that people can use it as a tool for ongoing generation and provisional testing of hypothesis about the video data.

[Tang, A., Greenberg, S., and Fels, S., 2008]

...our slit-tear tool conceptualizes the analysis space in a readily understood camera and pixel space. The timeline simply takes selected areas of the video and translates the time dimension to a spatial dimension. The raw data is still visible in this representation.

[Tang et al., 2008]

Evaluation

As per May 2009, there are no evaluations of slit-tear visualizations. Hence, an evaluation of the effectiveness of this tool for novice users in realistic tasks is planned for the near future.

References

[Tang et al., 2008] Anthony Tang, Saul Greenberg and Sidney Fels, 2008. Exploring Video Streams using Slit-Tear Visualizations. In Proceedings of the working conference on Advanced Visual Interfaces (AVI 2008). (May 28-30, Napoli, Italy). ACM Press. pp: 191-198.

Demonstrations

iLab Cookbook - Slit Tears: Visualizing Video over Time.