Exploring Video Streams using Slit-Tear Visualizations

Authors

• Anthony Tang
• Saul Greenberg
• Sidney Fels

Short description

Video slicing — a variant of slit scanning in photography — extracts a scan line from a video frame and successively adds that line to a composite image over time. The composite image becomes a time line, where its visual patterns reflect changes in a particular area of the video stream. We extend this idea of video slicing by allowing users to draw marks anywhere on the source video to capture areas of interest. These marks, which we call slit-tears, are used in place of a scan line, and the resulting composite timeline image provides a much richer visualization of the video data. Depending on how tears are placed, they can accentuate motion, small changes, directional movement, and relational patterns.

[Tang et al., 2008]

Suitable Datatypes

This method can be applied to live video as well as captured AVI files where each frame is either computed seperately or completely in advance, together with its successors. The latter is only relevant to non-live video.

Figures

Composition of a slit-tear visualization

For each video frame, the pictures under each slit are captured and aligned into a vertical column which is appended to the right side of the visualization. The bottom of the image represents a number of video frames that capture a top-down view of a room with three blue doorways. Herein, four slits have been manually defined. The actual slit-tear visualization above shows the occurence of events. In frame two, a person walks towards to the bottom door, crossing slit number 2. This intrusion is captured and can be seen as yellow trace in successive frames for the corresponding slit.

How slit-tears create a visualization

Pattern-level analysis

This image shows how events can be visualized spatially which allows for a discovery of patterns and for an effective correlation of events. Frames t1 to t5 capture a traffic intersection at particular moments in time according to the three slits, drawn by the observer. The interaction of cars and pedestrians is visualized by setting up a timeline for each pre-defined slit where the movement of the objects is depicted by strokes of different length, based on their orientation to each slit.

Traffic intersection

Important citations

More generally, events of interest can be problematic to see in conventional replay of video when the image quality may be poor, events may be very brief or spatially small, and patterns over time may be hard to detect. Slit-tears are a technique that helps to overcome these difficulties.

[Tang et al., 2008]

The interactivity of slit-tear placement at any time (inclding clearing old slit-tears) means that people can use it as a tool for ongoing generation and provisional testing of hypothesis about the video data.

[Tang et al., 2008]

...our slit-tear tool conceptualizes the analysis space in a readily understood camera and pixel space. The timeline simply takes selected areas of the video and translates the time dimension to a spatial dimension. The raw data is still visible in this representation.

[Tang et al., 2008]

Evaluation

As per May 2009, there are no evaluations of slit-tear visualizations. Hence, an evaluation of the effectiveness of this tool for novice users in realistic tasks is planned for the near future.

References

[Tang et al., 2008] Anthony Tang, Saul Greenberg and Sidney Fels, 2008. Exploring Video Streams using Slit-Tear Visualizations. In Proceedings of the working conference on Advanced Visual Interfaces (AVI 2008). (May 28-30, Napoli, Italy). ACM Press. pp: 191-198.

Demonstrations

iLab Cookbook - Slit Tears: Visualizing Video over Time.