Efficient Video Browsing - PowerPoint PPT Presentation

Efficient video browsing
1 / 32

  • Uploaded on
  • Presentation posted in: General

Efficient Video Browsing. Using Multiple Synchronized Views Heymo Kou. Question. What is the two main technologies applied for efficient video browsing? (one for audio, one for visual content). Table of contents. Background Current technology Advanced technology Summary Reference.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Efficient Video Browsing

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Efficient video browsing

Efficient Video Browsing

Using Multiple Synchronized Views

Heymo Kou



  • What is the two main technologies applied for efficient video browsing? (one for audio, one for visual content)

Table of contents

Table of contents

  • Background

  • Current technology

  • Advanced technology

  • Summary

  • Reference

Growth of digital contents data

Growth of digital contents data

Digital video market growth

Digital video market growth

Digital video becomes ubiquitous

Digital video becomes ubiquitous

  • From your

    • Smart phones

    • Notebooks

    • Webcams

    • Digital camera and camcorders

    • Security and monitoring cameras

  • With advanced streaming technology

    • Fast Internet access

    • MPEG-4 format

Current technology for finding a video data

Current technology for finding a video data

  • Search through categories

    • Similar to Internet shopping mall

      • We search for big categories

      • Then smaller categories

      • …and so on…

  • User should choose which to browse

    • Should check whether the selected data matches what user was finding

      • Time consuming!

  • Manual categorizing and annotation

    • One by one?

Problem with current video search and browsing technologies

Problem with current video search and browsing technologies

  • Too complicated

    • Lack of efficient algorithm

  • Time consuming

    • Multimedia calculation ∝ exponential

  • Inaccuracy

    • Video data is increasing exponentially

      • Cataloging manual has a somewhat limit point

    • Manually cataloging is done by human hand that mistakes can be happened

Technologies for advanced image and video retrieval

Technologies for advancedimage and video retrieval

  • MPEG-7 Standards

  • Speech indexing

  • Shot Boundary Detection

  • Time Scale Modification of Audio Signals

  • Storyboards, Moving Storyboards and Animation

  • Adaptive Accelerating Fast Playback

  • Streaming Synchronized Views

Mpeg 7 standard

MPEG-7 standard

  • Standardized by ISO/IEC

    • International Standard Organization

    • International Electrotechnical Commission

  • Not a video encoding format

  • XML to store metadata

    • Attached to timecode in multimedia

  • By this tag

    • Able to index and search efficiently

  • Yet, improvement is needed

Speech indexing

Speech indexing

  • Search through speech transcripts

    • Finds familiar metaphor of free text search

  • Automatic speech recognition (ASR)

    • Indexed transcript → semantic information

  • Main advantage : Representation

    • Speech is built of words



  • Frame

  • Key frame

  • Shot

    • Group of frameswhich represents similar frames

Start key frame end key frame animation



  • Context

    • Meaningful information within multimedia data

  • 3 levels of video browsing

    • Browsing a large collection of videos

    • Browsing a ranked list of videos

    • Browsing a single video to find relevant segments

Shot boundary detection

Shot Boundary Detection

  • Shot Boundary Detection(SBD) algorithm

    • Completely automatic

  • Key frames are selected and extracted

    • Saved as JPEG files

  • High Accuracy and Efficiency

    • Still, fault detection problem is unsolved

Sbd algorithm

SBD algorithm

Similar to scene selection of dvd

Similar to scene selection of dvd

Time scale modification of audio signals

Time Scale Modification ofAudio Signals

  • Audio browsing is as important asvideo browsing

    • Except images, most digital contents are audible

  • Faster audio browsing is necessary

  • Speeding up of audio signal by

    • By deleting small audio segments

    • Especially, human speech signals are quasi-periodic

Improvement of tsm

Improvement of TSM

Time-Domain Harmonic Scaling(TDHS) technique

Time-Domain, Pitch Synchronous Overlap Add

Time Scale Modification

(TSM) algorithm

Waveform Synchronous Overlap(WSOLA)

Synchronous overlap add sola

Synchronous Overlap-Add SOLA

Storyboards moving storyboards and animation

Storyboards, Moving Storyboards and Animation

  • Storyboard

    • a set of one or more pages, each consists of a two dimensional array of key-frames, sorted in chronological order.

  • Animation

    • a quick slide show, where each of the key-frames is shown for a fixed short period (e.g., 0.6 seconds)

  • Moving Storyboard (MSB)

    • the animated key frames, fully synchronized with the original audio track. Each key-frame is shown for the entire duration of the associated shot.

Adaptive accelerating fast playback

Adaptive Accelerating Fast Playback

  • Very fast video playback (without audio)

  • Ordinary fast forward depends only on speed

    • There is a chance to miss important scene

  • Accelerates until new scene is met

  • Requires less computation load

Image for adaptive fast playback

Image for adaptive fast playback

Real use of adaptive fast playback

Real-use of adaptive fast playback

  • Example in surveillance camera

Streaming synchronized views

Streaming Synchronized Views

  • Server preprocesses media

    • Keep same media, but different speed encoded

  • When user selects other speed

    • 1. pause current media

    • 2. open file with same content with selected speed

    • 3. seek to the corresponding position

    • 4. play the selected view

  • Needs no extra computational load

    • However, requires more storage: Tradeoff

Browsing multiple videos moviedna

Browsing Multiple Videos: MovieDNA

  • Can browse multiple videos at once

  • Split frames every given time

    • (i.e 10 seconds)

  • Strong information scent is visible

    • With aggregation of occurrences

Image of moviedna

Image of movieDNA

Summary of main properties

Summary ofmain properties



  • Streaming synchronized views and movieDNA

    • Less computation, multiple videos at once

  • Active accelerating fast playback

    • Most useful at analyzing surveillance videos

  • SBD & TSM

    • Efficient for implementing above technologies

  • Then, what is current limitation?

Efficient video browsing

Q & A

Any questions?



  • What is the two main technologies applied for efficient video browsing? (one for audio, one for visual content)

  • Answer : The two main technologies are Shot Boundary Detection(SBD) for visual content and Time Scale Modification(TSM) for audio signals

References 1 2

References (1/2)

  • Shot Boundary Detection

    • http://muvis.cs.tut.fi/sbd.html

  • Key frame

    • http://en.wikipedia.org/wiki/Key_frame

  • Synchronous Overlap-Add

    • http://www.surina.net/article/time-and-pitch-scaling.html

  • Digital Video Market Growth

    • http://articles.businessinsider.com/2011-12-13/research/30508929_1_fios-cable-providers-video-streaming-service/2

  • Amount of Digital data

    • http://www.emc.com/collateral/analyst-reports/diverse-exploding-digital-universe.pdf

References 2 2

References (2/2)

  • Streaming synchronized view

    • http://www.visus.uni-stuttgart.de/forschung/visualisierung-und-visual-analytics/visuelle-analyse-videostroeme/adaptive-fast-forward-for-video-surveillance.html

  • MovieDNA

    • http://homepage.mac.com/juggle5/WORK/publications/HICSS34.pdf

  • Login