1 / 28

Research and Future Perspectives on Intelligent Video Surveillance Systems

Research and Future Perspectives on Intelligent Video Surveillance Systems. Monique THONNAT Senior Scientist Head of Orion research team INRIA Sophia Antipolis FRANCE. Introduction 1/4. Which Security Problems? Safety and security of goods and human beings

trynt
Download Presentation

Research and Future Perspectives on Intelligent Video Surveillance Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Research and Future Perspectives on Intelligent Video Surveillance Systems Monique THONNAT Senior Scientist Head of Orion research team INRIA Sophia Antipolis FRANCE

  2. Introduction 1/4 Which Security Problems? • Safety and security of goods and human beings • Safety: protection in case of accident or incident (e.g.: fall) • Security: protection against a malicious act (ex.: bomb) • Evolution • 1980: guarding, human surveillance • 1990: start of video cameras setting up, remote surveillance • Highways, parking lots, subways, malls … • 2000: explosion of video-surveillance • Set of new laws • Better understanding of the general public • Threats of tragic events, terrorism acts

  3. Introduction: control room 2/4

  4. Introduction: issue 3/4 • Definition: intelligent video surveillance automatic analysis of the video streams • Why ? • Cost: 1 human operator /6-10 video streams • Current paradox : the more there are video cameras, the less these video camera are observed • E.g. 600 000 video cameras in London • need for the policemen to look at stored videos tapes • Effectiveness: duration of vigilance for an operator: • > 1 or 2hours: % of the attention is lost

  5. Video cameras Surveillance control rooms Video streams • Huge information flow • Few pertinent information Introduction 4/4 Video cameras • Selection of the information • Increase of the detection rate Video streams Intelligent video surveillance software Alarms Surveillance control rooms

  6. Intelligent Video Surveillance 1/3demonstration From object detection to complex event recognition (e.g.: violence)

  7. Intelligent Video Surveillance 2/3 Intelligent Video Surveillance Definition: • Data captured by video surveillance cameras • Real time and automated analysis of video sequences • Video understanding= from people detectionandtracking tobehavior recognition Recognition of complex behaviors: of individuals(e.g. fraud, graffiti, vandalism, bank attack) of small groups(e.g. fighting) of crowds(e.g. overcrowding) interactions of people and vehicles(e.g. aircraft refueling)

  8. Intelligent Video Surveillance 3/3Typical problems Metro station surveillance Surveillance inside trains Building access control Airport monitoring

  9. Video Understanding for Intelligent Video Surveillance 1/13 Definition: Cognitive Vision is a new research field mixing: • Computer vision techniques for object detection, description, categorization and tracking • Artificial intelligence techniques for knowledge acquisition, reasoning (e.g. spatial and temporal reasoning,…), learning (e.g. categories, structures, parameters…) • Software engineering techniques for vision software design, integration, reusability, evaluation Reference: http:www.eucognition.org/ecvision site and roadmap

  10. Video Understanding for Intelligent Video Surveillance 2/13 Intelligent Videosurveillance: How? • A Cognitive vision approach for video understanding mixing: • computer vision:4D analysis (3D + temporal analysis) • artificial intelligence:a priori knowledge (scenario, environment) • software engineering: reusable software platform (VSIP)

  11. Alarms Video Understanding for Intelligent Video Surveillance 3/13 Interpretation of the videos from pixels to alarms • A PRIORI KNOWLEDGE: • 3d models of the environment • Camera calibration • Scenario Models People detection and tracking 4 D analysis: multi-cameras tracking Scenario recognition People detection and tracking Video understanding

  12. Video Understanding for Intelligent Video Surveillance 4/13 Objective: Interpretation of videos from pixels to alarms Segmentation Classification Scenario Recognition Tracking Alarms access to forbidden area 3D scene model Scenario models A priori Knowledge

  13. Video Understanding for Intelligent Video Surveillance 5/13 • Behavior recognition: • approach based on a priori knowledge • model of the empty scene (3D geometry and semantics) • models of predefined scenarios • a language for representing scenarios based on combination of states and events • more than 20 states and 20 events can be used • a reasoning mechanism for real time detection of states, events and scenarios (e.g. temporal reasoning, constraints solving techniques)

  14. Video Understanding for Intelligent Video Surveillance 6/13 3D Scene Model:BarcelonaMetroStation Sagrada Famiglia mezzanine (European project ADVISOR)

  15. Video Understanding for Intelligent Video Surveillance 7/13 • States, Events and Scenarios : • State: a spatio-temporal property involving one or several actors on a time interval Ex : « close», « walking», « seated» • Event: asignificant change of states Ex : « enters», « stands up», « leaves » • Scenario: a long term symbolic application dependent activity Ex : « fighting», « vandalism»

  16. Video Understanding for Intelligent Video Surveillance 8/13Scenario Recognition : Temporal constraints • Vandalism scenario description : Scenario(vandalism_against_ticket_machine, Physical_objects((p : Person), (eq : Equipment, Name = “Ticket_Machine”) ) Components ((event s1: pmoves_close_toeq) (state s2: pstays_ateq) (event s3: pmoves_away_fromeq) (event s4: pmoves_close_toeq) (state s5: pstays_ateq) ) Constraints((s1 != s4) (s2 != s5) (s1 before s2) (s2 before s3) (s3 before s4) (s4 before s5) ) ) )

  17. Video Understanding for Intelligent Video Surveillance9/13 Vandalism in metro (Nuremberg, Germany)

  18. zone derrière le guichet armoire guichet zone de jour commode zone d’accès au bureau du directeur zone de jour/nuit zone d’entrée de l’agence porte salle automates zone devant le guichet salle automates porte d’entrée objet du contexte zone des distributeurs zone d’accès rue mur et porte salle du coffre rue Video Understanding for Intelligent Video Surveillance 10/13 3d Scene Model of 2 bank agencies

  19. Video Understanding for Intelligent Video Surveillance11/13 • Bank Monitoring:Bank attack scenario description: • scenarioBank_attack_one_robber_one_employee • physical_objects: • ((employee : Person), (robber : Person), z1: Back_Counter, • z2: Entrance_Zone, z3: Front_Counter, z4: Safe, d: Safe_door) • components: • (State c1 : Inside_zone(employee, z1)) • (Event c2 : Changes_zone(robber, z2,z3)) • (State c3 : Inside_zone(employee, z4)) • (State c4 : Inside_zone(robber, z4))) • constraints: • ((c2 during c1) (c2 before c3) • (c1 before c3) (c2 before c4) • (c4 during c3) • (d is open))

  20. Video Understanding for Intelligent Video Surveillance 12/13 bank monitoringRecognition of a bank attack scenario: an employee is e behind a counter, an aggressor enters, goes behind the counter, then he goes with the emplyee towards the STR (secured technical room), they enter in the STR then they leave ethe STR and they go toward the exit of the agency.

  21. Video Understanding for Intelligent Video Surveillance 13/13 • Examples : Brussels and Barcelona Metro Surveillance Group behavior Group behavior Blocking Fighting Exit zone Individual behavior Crowd behavior Jumping over barrier Overcrowding 21

  22. Conclusion 1/5 Impact: • Visual surveillance of metro stations, bank agencies, trains, buildings and airports • 5 European projects (PASSWORDS, AVS-PV, AVS-RTPW, ADVISOR, AVITRACK) • 4 contracts with End-users companies (metro, bank, trains) • 2 transfer activities with Bull (Paris) and Vigitec (Brussels) • Cooperation over more than 11 years with partners • Creation in 2005 of a start-up Keeneo www.keeneo.com

  23. Conclusion 2/5 • Hypotheses: • fixed cameras • 3D model of the empty scene • predefined behavior models • Results: + Behavior understanding for Individuals, Groups of people, Crowd or Vehicles + an operational language for video understanding (more than 20 states and events) + a real-time platform (10 to 25 frames/s)

  24. Conclusion 3/5 • Current issues • Systems have poorperformances over time, can be hardlymodified and do not use enoughaprioriknowledge strong perspective shadows tiny objects lighting conditions clutter close view

  25. Conclusion: Where we go 4/5 Knowledge Acquisition • Design of learning techniques to complement a priori knowledge: • Frequent events, scenario model learning • European project CARETAKER Object description • Fine human shape description:3D posture models • Crowd description: European project SERKET Reusability is still an issue for vision programs • Video analysis robustness • Dynamic configuration of programs and parameters

  26. Conclusion 5/5 Posture Recognition Current image and binary image Instantaneous posture Postures recognised along the time

  27. Conclusion 5/5 Crowd Behavior Motion direction detection Abnormal direction of people in a crowd

  28. Video Understanding demo? Airport Apron Monitoring “Unloading Operation” European AVITRACK project

More Related