using spatio temporal probabilistic framework for object tracking l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Using spatio-temporal probabilistic framework for object tracking PowerPoint Presentation
Download Presentation
Using spatio-temporal probabilistic framework for object tracking

Loading in 2 Seconds...

play fullscreen
1 / 21

Using spatio-temporal probabilistic framework for object tracking - PowerPoint PPT Presentation


  • 109 Views
  • Uploaded on

Using spatio-temporal probabilistic framework for object tracking. Emphasis on Face Detection & Tracking. By: Guy Koren-Blumstein Supervisor: Dr. Hayit Greenspan. Agenda. Previous research overview (PGMM) Under-segmentation problem Face tracking using PGMM

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Using spatio-temporal probabilistic framework for object tracking' - garron


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
using spatio temporal probabilistic framework for object tracking

Using spatio-temporal probabilistic framework for object tracking

Emphasis on Face Detection & Tracking

By: Guy Koren-Blumstein

Supervisor: Dr. Hayit Greenspan

agenda
Agenda
  • Previous research overview (PGMM)
  • Under-segmentation problem
  • Face tracking using PGMM
    • Modeling skin color in [L,a,b] color space – over-segmentation problem
  • Optical flow – overview
  • Approaches for using optical flow
  • Examples
previous research
Previous research
  • Complementary research to an M.Sc. Thesis research conducted by A.Mayer under the supervision of Dr. H.Greenspan and Dr. J. Goldberger.
  • Research Goal: Building a probabilistic framework for spatio-temporal video representation.
  • Useful for:
    • Offline – automatic search in video databases
    • Online – characterization of events and alerting on those that are defined as ‘suspicious’
previous research4
Previous research

Source Clip

BOF 1

Labeled BOF

Blob Extraction

Parsing clip to

BOF

Build feature

Space [L a b]

Build GMM model

In [Lab] space

Under segmentation problem…

Label BOF pixels

Connect. Comp.

On [L,a,b,x,y,t]

Learn GMM model

On [L,a,b,x,y,t]

face detection tracking
Face Detection & Tracking
  • Most of the known techniques can be divided into two categories :
    • Search for skin color and apply shape analysis to distinguish between facial and non-facial objects.
    • Search for facial features regardless of pixel color (eyes,nose,mouth,chin,symmetry etc.)
apply framework to track faces
Apply framework to track faces
  • The framework can extract and track after objects in an image sequence.
  • Applying shape analysis to each skin-colored-blob can label the blob as ‘face’ or ‘non-face’.
  • The face will be tracked by virtue of the tracking capabilities of the framework
skin color in l a b
Skin color in [L a b]
  • Skin color is modeled in [a b] components only
  • Supplies very good discriminability between ‘skin’ pixels and ‘not-skin’ pixels (high rate of True-Negative)
  • Not optimal in terms of True-Positive (leads to mis-detection of skin color pixels)
over segmentation of faces
Over-segmentation of faces
  • Building blobs is done in [L a b] color space.
  • More than one blob might have skin color [a b] components
  • Solution : Unite all blobs whose [a b] are close enough to the skin color model (adaptive TH can be used)
under segmentation
Under Segmentation
  • Faces moving in front of skin-color background are not extracted well.
  • Applying shape analysis on the middle map yields mis-detection of faces.
employing motion information
Employing motion information
  • Motion information helps to distinguish between foreground dynamic objects and static background
  • 2 levels of motion information
    • Binary – indicates for each pixel whether it is in motion or not. Does not supply motion vector. Feature space: [L a b x y t m] where m={0,1}
    • Optical flow - supplies motion vector according to a given model. Feature space: [L a b x y t Vx Vy]
optical flow
Optical Flow
  • Optical flow is an apparent motion of image brightness
  • If I(x,y,t) is the brightness, two main assumptions can be made:
    • I(x,y,t) depends on coordinates x,y in greater part of the image
    • Brightness of every point of moving object does not change in time
optical flow13
Optical Flow
  • If object is moving during time dt and its displacement is (dx,dy) then using Taylor series
  • According to assumption 2:
  • Dividing by dt gives the optical flow equation:
optical flow block matching
Optical Flow – Block Matching
  • Does not use the equation directly.
  • Divides the image to blocks
  • For every block in It it search for the best matching block in It-1.
  • Matching criteria: Cross Correlation, Square Difference, SAD etc.
working with 8 d feature space
Working with 8-D feature space

Parsing clip to

BOF

  • Connected component analysis:
    • Does not require initialization of the order of the model
    • Hard decision prone
  • GMM model via EM:
    • Initialized by K means. Requires initialization of K.
    • Impose elliptic shape on the objects
    • Soft Decision prone

Build feature

Space [L a b]

Build GMM model

In [Lab] space

Label BOF pixels

Connect. Comp.

On [x,y,t,Vx,Vy]

Learn GMM model

[x,y,t,Vx,Vy]

Frame By Frame

Tracking

frame by frame tracking
Frame by frame tracking

Label by predicted

parameters

  • Widely used in the literature
  • Can handle variations in object’s velocity
  • Tracking can be improved by employing Kalman filter to predict object’s location and velocity

Update blob’s params

Label by updated

parameters

Kill old blobs

split blobs

Create new blobs

merge blobs

Predict params for

next frame

examples
Examples
  • Opposite directions:
    • Optical Flow, Connected component (Extracted Faces), GMM
  • Same direction, different velocity
    • Optical Flow, Connected component, GMM (Faces)
  • Different directions – complex background
    • Optical Flow, Connected component, GMM: K=5,K=3,Faces
  • Variable velocity
    • Optical Flow, Connected component, GMM, Frame By Frame
real world sequences
Face tracking

Optical Flow

No motion info

Connected component

GMM

Frame By Frame

Car Tracking

Optical Flow

No Motion info

GMM

Flower garden

Optical Flow

No motion info

Connected component

GMM

Real World Sequences
summary
Summary
  • Applying probabilistic framework to track faces in video clips
  • Working in [L,a,b] color space to detect faces
  • Handling over segmentation
  • Handling under segmentation by employing optical flow information in 3 different ways:
    • Connected Component Analysis
    • Learning GMM model
    • Frame By Frame tracking
further research
Further Research
  • Adaptive face color model
  • Variable length BOF (using MDL)
  • Using more complex motion model