Context dependent detection of unusual events in videos by geometric analysis of video trajectories
This presentation is the property of its rightful owner.
Sponsored Links
1 / 40

Longin Jan Latecki ( [email protected] ) Computer and Information Science s PowerPoint PPT Presentation


  • 72 Views
  • Uploaded on
  • Presentation posted in: General

Context-dependent Detection of Unusual Events in Videos by Geometric Analysis of Video Trajectories. Longin Jan Latecki ( [email protected] ) Computer and Information Science s Temple University, Philadelphia Nilesh Ghubade and Xiangdong Wen ( [email protected] ). Agenda. Introduction

Download Presentation

Longin Jan Latecki ( [email protected] ) Computer and Information Science s

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Context dependent detection of unusual events in videos by geometric analysis of video trajectories

Context-dependent Detection of Unusual Events in VideosbyGeometric Analysis of Video Trajectories

Longin Jan Latecki

([email protected])

Computer and Information Sciences

Temple University, Philadelphia

Nilesh Ghubade and Xiangdong Wen

([email protected])


Agenda

Agenda

  • Introduction

  • Mapping of video to a trajectory

  • Relation: motion trajectory  video trajectory

  • Discrete curve evolution

  • Polygon simplification

  • Key frames

  • Unusual events in surveillance videos

  • Results


Main tools

Main Tools

  • Mapping the video sequence to a polyline in a multi-dimensional space.

  • The automatic extraction of relevant frames from videos is based on polygon simplification by discrete curve evolution.


Mapping of video to a trajectory

Mapping of video to a trajectory

  • Mapping of the image stream to a trajectory (polyline) in a feature space.

  • Representing each frame as:

………

X-coord of

the Bin’s

centroid

Y-coord of

the Bin’s

centroid

Bin’s

Frequency

Count

Frame 0

Bin0

Bin n

Frame N

Bin n


Used in our experiments

Used in our experiments

  • Red-Green-Blue (rgb) Bins

    • Each frame as a 24-bit color image (8 bit per color intensity):

      • Bin 0 = color intensities from 0-31

      • Bin 1 = color intensities from 32-63

      • Bin 8 = color intensities from 224-255

    • Three attributes per bin: -

      • Row of the bin’s centroid

      • Column of the bin’s centroid

      • Frequency count of the bin.

    • (8 bins per color level * 3 attributes/bin)*3 color levels = 72 feature


Longin jan latecki latecki temple computer and information science s

Theoretical Results:

Motion trajectory  Video trajectory

Consider a video in which an object (a set of pixels) is moving on a uniformbackground. The object is visible in all framesand it is moving with a constant speed on a linear trajectory.Then the video trajectory in the feature space is a straight line.

If n objects are moving with constant speeds on a linear trajectory,then the trajectory is a straight line in the feature space.


Longin jan latecki latecki temple computer and information science s

Consider a video in which an object (a set ofpixels) is moving on a uniform background.

Then the trajectoryvectors are containedin the plane.

If n objects are moving, then the dimension of the trajectory is at most 2n.

If a new object suddenly appears in the movie, the dimension of the trajectory increases at least by 1 and at most by 3.


Longin jan latecki latecki temple computer and information science s

MovingDotMovieWithAdditionalDot.avi


Robust rank computation

Robust Rank Computation

Using singular value decomposition, based on:

C. Rao, A. Yilmaz, and M.Shah.View-Invariant Representation and Recognition of actions.Int. J. of Computer Vision 50, 2002.

M. Seitz and C. R. Dyer.View-invariant analysis of cyclic motion.

Int. J. of Computer Vision 16, 1997.

We compute err in a window of 11 consecutive frames in our experiments.


Longin jan latecki latecki temple computer and information science s

MovingDotMovieWithAdditionalDot.avi


Interpolation of video trajectory

Interpolation of video trajectory

MovingDotMovie_Clockwise.avi


Longin jan latecki latecki temple computer and information science s

MovingDotMovieWithAdditionalDot.avi


Polygon simplification

Polygon simplification

Relevance Ranking

Frame Number

0

1

1

100

Frames with decreasing relevance

98

12

99

5


Longin jan latecki latecki temple computer and information science s

Discrete Curve Evolution P=P0, ..., PmPi+1 is obtained from Pi by deleting the vertices of Pi that have minimal relevance measure K(v, Pi) = K(u,v,w) = |d(u,v)+d(v,w)-d(u,w)|

v

v

w

w

u

u


Discrete curve evolution preservation of position no blurring

Discrete Curve Evolution: Preservation of position, no blurring


Discrete curve evolution robustness with respect to noise

Discrete Curve Evolution: robustness with respect to noise


Discrete curve evolution extraction of linear segments

Discrete Curve Evolution: extraction of linear segments


Key frame extraction

Key Frame Extraction


Key frames and rank

Key frames and rank

Security1

  • Bins Matrix

  • Distance Matrix


Longin jan latecki latecki temple computer and information science s

err for seciurity1 video


M s drew and j au http www cs sfu ca mark ftp acmmm00

M. S. Drew and J. Au: http://www.cs.sfu.ca/~mark/ftp/AcmMM00/


Predictability of video parts local curveness computation

Predictability of video parts:Local Curveness computation

We divide the video polygonal curve P into parts T_i. For videos with 25 fps:T_i contains 25 frames.

We apply discrete curve evolution to each T_iuntil three points remain: a, b, c.Curveness measure of T_i:

C(T_i,P) = |d(a, b) + d(b, c) - d(a, c)|

b is the most relevant frame in T_i

and the first vertex of T_i+1


Longin jan latecki latecki temple computer and information science s

security7


Longin jan latecki latecki temple computer and information science s

err for

seciurity7


Longin jan latecki latecki temple computer and information science s

2D projection by PCA of video trajectory for security7


Longin jan latecki latecki temple computer and information science s

Mov3


Longin jan latecki latecki temple computer and information science s

Mov3:

Rustam waving his hand.

  • Bins Matrix

    Keyframes = 1 378 52 142 253 235 148 31 155 167

  • Distance Matrix

    Keyframes = 1 378 253 220 161 109 50 155 149 270


Longin jan latecki latecki temple computer and information science s

Hall_monitor


Longin jan latecki latecki temple computer and information science s

err for

hall_monitor


Longin jan latecki latecki temple computer and information science s

Hall Monitor:

2 persons entering-exiting in a hall.

  • Bins Matrix

    Keyframes = 1 300 35 240 221 215 265 241 278 280

  • Distance Matrix

    Keyframes = 1 300 37 265 241 240 235 278 280 282


Longin jan latecki latecki temple computer and information science s

CameraAtLightSignal.avi


Multimodal histogram

Multimodal Histogram

Histogram of lena


Segmented image

Segmented Image

Image after segmentation – we get a outline of her face, hat etc


Gray scale image multimodal

Gray Scale Image - Multimodal

Original Image of Lena


Longin jan latecki latecki temple computer and information science s

Thank you


  • Login