1 / 21

Multi-Scale Video Cropping

Multi-Scale Video Cropping. Hazem El-Alfy , David Jacobs and Larry Davis Department of Computer Science University of Maryland, College Park Sep 25 th 2007, ACM MM ’07. Modern Surveillance Systems. Networks of sur-veillance cameras. Control Room: Fewer monitors than cameras.

bowlesc
Download Presentation

Multi-Scale Video Cropping

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Multi-Scale Video Cropping Hazem El-Alfy, David Jacobs and Larry Davis Department of Computer Science University of Maryland, College Park Sep 25th 2007, ACM MM ’07

  2. Modern Surveillance Systems • Networks of sur-veillance cameras. • Control Room: • Fewer monitors than cameras. • Far fewer operators than monitors. • Cameras cycle through monitors.

  3. Modern Surveillance Systems Typical Control Rooms: airports, subways, metropolitan areas, seaports, crowd control.

  4. “Future” Control Rooms • “Continuous” display wall versus a fixed set of discrete monitors. • Algorithms to control: • where to display videos, • how much area to assign to them, • how to display them. Barco Control Room, Vienna, Austria

  5. Video Cropping Munich Airport – Courtesy Siemens, NJ

  6. Why Cropping? • Resize video to save bandwidth or to fit display area. • Cropping before resizing to focus operator attention of on important areas.

  7. y t x Problem Definition Determine trajec-tories of cropping windows through the video: • variable size window • maximize captured saliency • smooth trajectory • occasional jumps (cuts) between trajectories.

  8. Problem Definition • Each frame t covered by variable size overlapping windows Wi,t • Saliency measure S(Wi,t) • argmaxQΣtS(Wi,t), over all window sequences Q • Subject to constraints for smooth window motion and size change. Wi,t

  9. Our Approach: Overview • Extract motion energy. • Model video as a graph. • Find trajectories as shortest paths in graph. • Merge trajectories. • Repeat for other segments of long videos.

  10. Extracting Motion Energy • Motion energy as a saliency measure. • Frame differences are smoothed using morphological operations.

  11. Modeling Graph • Nodes: cropping windows in each frame. • Add dummy source and target nodes. • Edges: allowable window changes (location and size) between consecutive frames. w=0 dummy source node dummy target node w=0 windows of first frame windows of i th frame windows of last frame

  12. Modeling Graph • Multi-scale energy function for window W: • E(W) = S(W): always favors large windows • E(W) = S(W)/A(W): favors small (dense) windows • E(W) = S(Win)/A(Win) – Sbelt/K • Edge weight: wij = 1 – ENorm(Wj)

  13. x2 x1 cropping window W x4 x3 video frame Modeling Graph • Energy function computed for all windows in all frames. • Efficiently computed using integral images [Viola & Jones ’01]: • ii(x,y) = Σx’<x,y’<yi(x’,y’) • E(W)=ii(x3)-ii(x2)-ii(x4)+ii(x1)

  14. Shortest Path • Dial’s implementation of Dijkstra’s algorithm: linear in # graph nodes. • Smoothing: low-pass filter + cubic Hermite interpolation.

  15. Merging Trajectories • More cropping windows needed to capture simultaneous activity. • Wipe captured activity from motion frames and repeat earlier process on remaining motion. • Merge trajectories: find shortest path through a graph of trajectories.

  16. Processing Long Videos • Problems: • Graph gets too big if video is long. • Latencies must be short in surveillance systems. • Solution: • Break long videos into segments with overlap. • Process each segment then stitch results together. break here break here

  17. Processing Long Videos • Issues • How short can segments be? • Are there preferable locations to break video? • Overlap amount needed for smooth transitions? • We ran many experiments for fixed size crop • Shortest path converge quickly. Segments can be as short as 40 frames. • Avoid periods of low activity when breaking video. • Overlap intervals of 20 frames are sufficient.

  18. Results Munich Airport: variable size single window.

  19. Results Munich Airport: video-in-video display.

  20. Results Traffic at a stop sign on campus (2 windows).

  21. Contributions • Variable size smooth cropping window. • Simultaneous multiple cropping windows. • Relatively short video segments processed vs. the entire video (online). • Empirically shown identical to processing the largest video that can be processed as a whole.

More Related