1 / 48

Video Classification

Video Classification. By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project. Outline. What is Video Classification? Straightforward or Difficult? What is its Applications? What are its methods? Review of Video Classification Methods

Download Presentation

Video Classification

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Video Classification By: Maryam S. Mirian For: Multimedia & Pattern Recognition Joint Courses Project

  2. Outline • What is Video Classification? • Straightforward or Difficult? • What is its Applications? • What are its methods? • Review of Video Classification Methods • What is my own Project, exactly?

  3. What is Video Classification? Classify a Video (Shot) into one of Nc predefined Classes: • Indoor / outdoor • News / Sports • …

  4. Is Video Classification Difficult? Why? • YES, Because: • Data Stream is a Multi-dimensional signal. • It has a subjective nature.

  5. Classification

  6. Required Steps for Classification Object Observations Feature Extraction Feature Reduction Classification Class Labels Using Methods like: PCA, LDA The most Important and the most difficult part

  7. Methods of Classification • Bayesian Classification • kNN Classification • Neural Classification • MLP • RBF • Classification based on Support Vector Machines • Rule-based Classification

  8. Bayesian Decision Making So, x belongs to w2

  9. Methods of Classification • Bayesian Classification • kNN Classification • Neural Classification • MLP • RBF • Classification based on Support Vector Machines • Rule-based Classification

  10. kNN Decision Making k = 5, 2 Red Neighbor While 3 Black Neighbor, so X should be Black!

  11. Methods of Classification • Bayesian Classification • kNN Classification • Neural Classification • MLP • RBF • Classification based on Support Vector Machines • Rule-based Classification

  12. MLP Classifier

  13. Video Content Analysis

  14. Applications ofAutomatic video classification • Automatic Video segmentation • content based retrieval • browsing and retrieving digitized video • identifying close-up video frames before running a computationally expensive face recognizer. • effective management of ever-increasing amount of broadcast news video: personalization of news video.

  15. Classify Shot or Video? • One effective way to organize the video is to segment the video into small, single-story units and classify these units according to their semantics. • A shot represents a contiguous sequence of visually similar frames. It is a syntactical representation and does not usually convey any coherent semantics to the users.

  16. Looking @Video Classification

  17. Ide et al. [1998] • Problem Domain: News video • Features: • Videotext • motion • face • segmented the video into shots • used clustering techniques • classify each shot into 1 of 5 classes: Speech/report, Anchor, Walking, Gathering, and Computer graphics shots. • Quite simple but seems effective for this restricted class of problems.

  18. Huang et al. [1999] • Problem Domain: TV Programs • news report • weather forecast • Commercials • basketball games • football games • Features: • Audio • Color • motion

  19. Chen and Wong [2001] • Problem Domain: • news video: • News • Weather • Reporting • Commercials • Basketball • Football • Features: • Motion • Color • text caption • cut rate • used a rule-based approach

  20. Looking @ Lekha Chaisorn et.al [2002] in More Details

  21. Basic Ideas • Proposes a two-level, multi-modal framework. • The video is analyzed at the shot and story unit (or scene) levels. • At the shot level, a Decision Tree to classify the shot into one of 13 pre-defined categories is employed. • At the scene level, the HMM (Hidden Markov Models) analysis is used to eliminate shot classification errors • Results indicate that a high accuracy of over 95 % for shot classification can be achieved. • The use of HMM analysis helps to improve the accuracy of the shot classification and achieve over 89% accuracy on story segmentation.

  22. Predefined Classes

  23. Features in Shot Level • Low-level Visual Content Feature • Color Histogram • Temporal Features • Background scene change • Speaker change • Audio • Motion activity • Shot duration • High-level Object-based features • Face • Shot type • Videotext • Centralized Videotext

  24. Feature vector of a shot • Si = (a, m, d, f, s, t, c) • a the class of audio, a ∈{ t=speech, m=music, s=silence, n =noise, tn = speech + noise, tm= speech + music, mn=music+noise} • m the motion activity, m ∈{l=low, m=medium, h=high} • d the shot duration, d ∈{s=short, m=medium, l=long} • f the number of faces, Ν ∈ f • s the shot type, s ∈{c= closed-up, m=medium, l=long, u=unknown} • t the number of lines of text in the scene, Ν ∈ t • c set to “true” if the videotexts present are centralized, c ∈{t=true, f=false}

  25. Decision Tree for Shot Classification

  26. Reading these papers, I decided about My own Project….

  27. About Problem Domain… • Sport Classification seems OK • Interesting Enough • It is helpful for Sports-Lovers

  28. About Extracting features…. • Features used in video analysis: color,texture,shape,motion vector… • Criteria of choosing features : they should have similar statistical behavior across time • Color histogram: simple and robust • Motion vectors:invariance to color and light

  29. So, My Own Project is • Sports Video Classifications : Football, Basketball, ….(Those Well-defined sports, I can find Video On!) • Steps I should take: • Finding or Gathering a Video Collection • Shot Detection • Feature Extraction : • Key Frame (s) Extraction: • Selecting Middle Shot I-Frame • Use of Clustering • … • Motion Vector–based Features • Straight Lines Detection • Design a Classifier • Test the Approach

  30. Looking @Ekin,Tekalp[2003] one Research on Football Video Classification

  31. Features • Cinematic • result from common video composition and production rules. • shot types, camera motions and replays. • Object-based • Described by their spatial, e.g., color, texture, and shape, and spatio-temporal features, such as object motions and interactions

  32. Robust Dominant Color Region Detection • A soccer field has one distinct dominant color (a tone of green) that may vary from stadium to stadium, and also due to weather and lighting conditions within the same stadium. • The statistics of this dominant color, in the HSI space, are learned by the system at start-up, and then automatically updated to adapt to temporal variations.

  33. Shot classification • Long Shot • A long shot displays the global view of the field. • In-Field Medium Shot • a whole human body is usually visible. • Close-Up Shot • shows the above-waist view of one person • Out of Field Shot • The audience, coach, and other shots

  34. How Extend to Shot from a Frame? • Due to the computational simplicity they find the class of every frame in a shot and assign the shot class to the label of the majority of frames.

  35. Decision Schema based on G • The first stage uses G value and two thresholds,TcloseUpand Tmedium to determine the frame view label.

  36. Soccer Eevent Detection • Goal Detection • Referee Detection • Controversial calls, such as red-yellow cards and penalties • Penalty Box Detection

  37. Goal Detection • Occurrence of a goal is generally followed by a special pattern of cinematic features. • A goal event leads to a break in the game. • one or more close-up views of the actors of the goal event. • show one or more replay(s) • the restart of the game is usually captured by a long shot.

  38. Referee Detection • Assumed that there is, a single referee in a: • medium • out of field • close-up shot • So no search for a referee in a long shot

  39. Penalty Box Detection • Field lines in a long view can be used to localize the view and/or register the current frame on the standard field model

  40. Interesting Summaries • Goal summaries • summaries with Referee and Penalty box objects

  41. Adaptation of Parameters • Parameters • Tcolor in dominant color region detection • TcloseUp and Tmedium in shot classification • referee color statistics • The training stage can be performed in a very short time to find Mean and Variance of a Normal pdf.

  42. Results for High-Level Analysis and Summarization • Goal detection results

  43. Results for High-Level Analysis and Summarization(2) • Referee detection results

  44. Results for High-Level Analysis and Summarization(3) • Penalty box detection results

  45. References • Automatic soccer video analysis and summarization, in Symp. Electronic Imaging: Science and Technology: Storage and Retrieval for Image and Video Databases IV, IS&T/SPI03, Jan. 2003, CA. • “The Segmentation and Classification of Story Boundaries In News Video”, Proceeding of 6th IFIP working conference on Visual Database Systems- VDB6 2002, Australia 2002 • Pattern Classification, by Duda, Hart, and Stork, 2000

  46. Thanks for Your Attention Any Question or Comment?

More Related