Background Removal

1. David Harwin Adviser: Petros Faloutsos Background Removal

2. The Problem • Many applications use video to record data • However, lots of data is wasted on representing the background, which can be unnecessary at best and distracting at worst • Removing background can improve clarity and reduce file size

3. The Solution • Capture an image of the background with no foreground images • Process each subsequent frame of video by comparing it to the basis image • Portions of the image determined to be in the background are replaced with arbitrary data • Use a scene graph to include higher-level processing to improve accuracy

4. Applications and Goals • Medical recordings: removing extraneous distractions • quality of foreground data is critical • not constrained to real-time processing • Visual effects processing: simpler or alternative to chroma key • quality is important • allows long processing time • Videoconferencing: improve performance of video compression techniques (interframe and intraframe)‏ • some artifacts are acceptable • requires real-time or near-real-time processing • For this project, the goal is accuracy and preservation of foreground data

5. Difficulties • Must allow some motion in the background (caused by wind, etc.)‏ • Must allow slight lighting changes in the background • Must keep tolerances high enough to preserve foreground

6. Tasks Involved • Set up environment for handling and processing video (cross-platform is a plus)‏ • Implement a background removal algorithm • Compare with existing implementations • Extend to stereo, adding 3D geometry, converting 2D to 3D, etc.

7. Deliverables • Project length: 2 quarters • Checkpoint 1: • Have framework for video input and processing • Checkpoint 2: • Have background removal implementation • Have comparison with existing implementations