Introduction of computer vision
This presentation is the property of its rightful owner.
Sponsored Links
1 / 52

Introduction of Computer Vision PowerPoint PPT Presentation


  • 98 Views
  • Uploaded on
  • Presentation posted in: General

Introduction of Computer Vision. Pedestrian Detection. Finding People in Images and Videos Navneet DALAL http://lear.inrialpes.fr/people/dalal/NavneetDalalThesis.pdf Chapter 4: Histogram of Oriented Gradients Based Encoding of Images http://lear.inrialpes.fr/pubs/2005/DT05/cvpr2005_talk.pdf

Download Presentation

Introduction of Computer Vision

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Introduction of computer vision

Introduction of Computer Vision


Pedestrian detection

Pedestrian Detection

  • Finding People in Images and Videos

    • Navneet DALAL

    • http://lear.inrialpes.fr/people/dalal/NavneetDalalThesis.pdf

    • Chapter 4: Histogram of Oriented Gradients Based Encoding of Images

    • http://lear.inrialpes.fr/pubs/2005/DT05/cvpr2005_talk.pdf

    • Histograms of Oriented Gradients for Human Detection http://lear.inrialpes.fr/pubs/2005/DT05/hog_cvpr2005.pdf


Datasets

Datasets

  • INRIA Person Dataset

    • http://pascal.inrialpes.fr/data/human/

  • Computer Vision Datasets

    • http://clickdamage.com/sourcecode/cv_datasets.html


Today s topic

Today’s Topic

  • Motion Detection


Motion detection

Motion Detection

  • Temporal Differencing

    • Take the difference between two temporally adjacent frames. The difference is the moving pixels (almost). The static background results in zeros.

    • Can adapt to changing lighting conditions because the difference frames are only 1/30 of a second apart (typical video 30 frames per second – 30 fps)


Motion detection1

Motion Detection

  • Temporal Differencing Issues

    • Not all the relevant pixels extracted

    • Background pixels extracted.


Motion detection2

Motion Detection

Frame at time t Frame at time t+1 Frame Difference

Red Block appears

as two separate objects


Sidewalk scene

Sidewalk Scene


Motion detection difference method

Motion DetectionDifference Method

Sidewalk 12_5


Processing video in matlab

Processing Video in Matlab

%C:\Program Files\MATLAB\R2008b\toolbox\OCR\BackgroundAnalysis02_02_2010

bkg = 20; %frames of video to be processed

fname = 'Office1.avi';

vidObj = mmreader(fname);

%Play video

implay('Office1.avi');

nFrames = vidObj.NumberOfFrames;

rw = vidObj.Height;

cl = vidObj.Width;

numFrames = 1000;

CODE TO PROCESS FRAMES HERE


Writing frames to directory

Writing Frames to Directory

imwrite(objBox,['.\video\','LabScene','.',num2str(i),'.jpg'],'jpg');

imwrite(objBoxVideo,['.\video\','LabSceneColor','.',num2str(i),'.jpg'],'jpg');


Matlab vidobj mmreader fname

MATLAB vidObj = mmreader(fname);

%motionDet.m

fname = 'sidewalk11_23indeo.avi';

a = aviread(fname); %%OLD METHOD

frameInfo = aviinfo(fname);

totalFrames = frameInfo.NumFrames

for i = 1:50

%for i = 1:totalFrames-1

currentFrameDiff = abs(im2double(a(1,i+1).cdata)-im2double(a(1,i).cdata));

movDiff(i) = im2frame (currentFrameDiff);

end

%MATLAB Movie file

figure, movie(movDiff)

% FOR AVI MOVIE

%movie2avi(movDiff,'sidewalk12_05_07.avi','compression', 'none');


Processing avi files

Processing .avi files

%readWriteAviFiles

fname = 'CarsTarget2.avi';

% extracting the frame information.

%frameInfo = aviinfo( strcat( pathname, fname ));

frameInfo = aviinfo( fname );

disp( frameInfo );

for cnt = 1:20

mov1=aviread(fname,cnt);

frame1 = mov1(1,1).cdata; %uint8

image1= im2double(frame1);

figure,imshow(image1);

%WRITE INDIVIDUAL FRAMES TO DIRECTORY

imwrite(image1,['.\video\','CarVideo','.',num2str(cnt),'.jpg'],'jpg');

end


Create video from individual frames

Create Video from Individual Frames

  • VirtualDub


Motion detection3

Motion Detection

  • Background Modeling

    • Model background without moving objects

      • Represent each pixel in the frame with a 3D Gaussian – mean red, green, blue and covariance matrix

      • For each pixel, collect n pixel triplets.

      • Use triplets to estimate mean and covariance matrix

      • Process future frames by determining the probability of each pixel in the new frame

      • Threshold the probability, p(r,c)>thres is a foreground pixel (moving object)

    • Compare pixel values in current frame and estimate if pixel is represented by background distribution or more likely from a different distribution (therefore new object not in background)


Updating gaussian distributions

Updating Gaussian Distributions

  • Small changes in the environment will result in thresholding errors

  • Adapt the Gaussian models by calculating a weighted average

    • Estimate means and covariance matrix from initial frames

    • Update distributions using pixels identified as background – distributions will adjust for slight changes in lighting conditions


Introduction of computer vision

  • Instead of using estimated covariance matrix use the identity matrix

    • How does this change affect performance??


Pixel modeling stationary camera

Pixel ModelingStationary Camera

Sidewalk Threshold


Overpass

Overpass

Object Tracking Overpass


Face tracking example all objects

Face Tracking Example – All Objects


Face tracking example faces only

Face Tracking Example – Faces Only


Non stationary camera

Non-stationary Camera

  • Example: A camera panning a scene

  • One approach is to register the adjacent frames

    • Find key points in adjacent frames

    • Determine offset

    • Adjust images so that they overlap

    • Take difference


Need to find correspondence between feature points in two different images

NEED TO FIND CORRESPONDENCE BETWEEN FEATURE POINTS IN TWO DIFFERENT IMAGES

  • Cannot match individual pixels

  • Need to use a window containing many pixels (5x5, 7x7, 21x21, etc)


Introduction of computer vision

  • Matching on a continuum like texture and edges not very robust

    • Many edges (and parts of edges) will match

  • At the very least:

    • Need to find interest points

    • Extract patches around interest points

    • Match patches


Feature points correspondence

Feature Points / Correspondence

  • Points should be extracted consistently over different views

  • Points should be invariant to scaling, rotation, changes in illumination

  • Information in the neighborhood of the point should be unique so that it can be matched


Select window region match region in second image

Select Window RegionMatch Region in Second Image

Calculate difference between the two patches

WindowMatching_ACV.m


Convert to grayscale

Convert to Grayscale


Randomly select patch

Randomly Select Patch

patch = I(80:110,200:230);

For Demonstration Use Only Strip Containing Patch


Error absolute difference between patch and strip

Error=Absolute Difference Between Patch and Strip


Normalized cross correlation refer to equation on page p313 also matlab docs

Normalized Cross Correlation (Refer to equation on page p313)Also MATLAB docs

  • w is template

  • w is average value of elements in template

  • f is the image

  • f is the average of the image where f and w overlap

  • Denominator normalizes resulting in an output range of -1, +1

  • High value for absolute value of output is a good match


Matlab cross correlation function g normxcorr2 template f

MATLAB Cross Correlation Function g = normxcorr2(template, f)


Find max value in g

Find Max Value in |g|

d = abs(g);

[ypeak, xpeak] = find(d == max(d(:)));

%Adjust location by size of template

ypeak = ypeak-(size(patch,1)-1)/2;

xpeak = xpeak-(size(patch,2)-1)/2;

fprintf('\n Center of Patch: ypeak is

%d and xpeak is %d \n\n\n', ypeak, xpeak);

figure, imshow(Igray)

hold on

plot(xpeak, ypeak, 'ro')


Introduction of computer vision

Red – strongest response

Green – second strongest response


But how do we select points

But How Do We Select Points?

  • Junctions or Corners

  • Stable over changes in viewpoint


Feature points

Feature Points?


Subtract first window from second window

Subtract First Window from Second Window


Moravec s corner detector

Moravec’s Corner Detector

  • Overview:

    • Select window size

    • Shift window over image region

      • If window over uniform region, shifts in all directions will result in small changes

      • If window over edge, shifts along edge will results in small changes, but shifts across edge will result in large changes

        • along edge – no change

        • Perpendicular to edge – large change

      • If window over corner, than shifts in all directions will result will result in large changes

  • Detect corner by finding regions that have large changes in all directions


Subtract first window from second window1

Subtract First Window from Second Window

Window moved vertically, no change

Window moved horizontally, no change

Window moved in either

direction, large change


Harris points

Harris Points

Corner Response function, C:

C = det(A) – αtrace2(A), where A is the autocorrelation matrix

.

Fig. 1: Autocorrelation matrix,

where w(x, y) is the window function and I(x, y) is the image

REF: image from Wikipedia


Corner detector

Corner Detector


Rotated image

Rotated Image


Non stationary camera1

Non-stationary Camera

  • Example: A camera panning a scene

  • One approach is to register the adjacent frames

    • Find key points in adjacent frames

    • Determine offset

    • Adjust images so that they overlap

    • Take difference


Panning a building complex

Panning a Building Complex


Two sequential frames color

Two Sequential Frames - Color

What if you just simply take the difference between two adjacent frames?


Two sequential frames grayscale

Two Sequential Frames -Grayscale


Abs frame11 frame10

abs(frame11-frame10)


Overall approach

Overall approach


Points of interest

Points of Interest


Correspondence

Correspondence


Difference

Difference


  • Login