Raffay Hamid, Amos Johnson, Samir Batta, Aaron Bobick, Charles Isbell, Graham Coleman - PowerPoint PPT Presentation

oshin
raffay hamid amos johnson samir batta aaron bobick charles isbell graham coleman l.
Skip this Video
Loading SlideShow in 5 Seconds..
Raffay Hamid, Amos Johnson, Samir Batta, Aaron Bobick, Charles Isbell, Graham Coleman PowerPoint Presentation
Download Presentation
Raffay Hamid, Amos Johnson, Samir Batta, Aaron Bobick, Charles Isbell, Graham Coleman

play fullscreen
1 / 47
Download Presentation
Raffay Hamid, Amos Johnson, Samir Batta, Aaron Bobick, Charles Isbell, Graham Coleman
316 Views
Download Presentation

Raffay Hamid, Amos Johnson, Samir Batta, Aaron Bobick, Charles Isbell, Graham Coleman

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Activity Discovery and Anomalous Activity Explanation Raffay Hamid, Amos Johnson, Samir Batta, Aaron Bobick, Charles Isbell, Graham Coleman

  2. Activity Discovery and Anomalous Activity Explanation

  3. Activity Discovery and Anomalous Activity Explanation • Anomaly - “deviation” from the “common” or “regular”

  4. Activity Discovery and Anomalous Activity Explanation • Anomaly - “deviation” from the “common” or “regular” • Key Questions: • ‘representation’ of activities • ‘regular’ activities • ‘different’ from regular • ‘explain’an anomaly

  5. Activity Discovery and Anomalous Activity Explanation • Anomaly - “deviation” from the “common” or “regular” • Key Questions: • ‘representation’ of activities • ‘regular’ activities • ‘different’ from regular • ‘explain’an anomaly

  6. Activity Representation • Previous representations include: • Stochastic Context Free Grammars • Expectation Grammars • …..

  7. Activity Representation • Previous representations include: • Stochastic Context Free Grammars • Expectation Grammars • ….. • Require some a priori information about activity structure

  8. Activity Representation • Two pieces of information: • content • structure • Drawing from Natural Language Processing – treating documents as bags of words • Treat Activities as bags of event n-grams • Extraction of global structural information using local event statistics

  9. Activity Representation • Two pieces of information: • content • structure • Drawing from Natural Language Processing – treating documents as bags of words –captures content well • Treat Activities as bags of event n-grams • Extraction of global structural information using local event statistics

  10. Activity Representation • Two pieces of information: • content • structure • Drawing from Natural Language Processing – treating documents as bags of words –captures content well • Treat Activities as bags of event n-grams –captures activity structure • Extraction of global structural information using local event statistics

  11. Activity Representation • Two pieces of information: • content • structure • Drawing from Natural Language Processing – treating documents as bags of words –captures content well • Treat Activities as bags of event n-grams –captures activity structure • Extraction of global structural information using local event statistics

  12. Activity Discovery and Anomalous Activity Explanation • Anomaly - “deviation” from the “common” or “regular” • Key Questions: • ‘representation’ of activities • ‘regular’ activities • ‘different’ from regular • ‘explain’an anomaly

  13. Activity Discovery and Anomalous Activity Explanation • Anomaly - “deviation” from the “common” or “regular” • Key Questions: • ‘representation’ of activities • ‘regular’ activities • ‘different’ from regular • ‘explain’an anomaly - Occur frequently - Are similar to each other

  14. Activity Discovery and Anomalous Activity Explanation • Anomaly - “deviation” from the “common” or “regular” • Key Questions: • ‘representation’ of activities • ‘regular’ activities • ‘different’ from regular • ‘explain’an anomaly - Activity similarity - Activity discovery

  15. Activity Similarity • Two types of differences • core structural differences (csd) • event frequency differences (efd) • Sim (A,B) = w1*CSD(A,B) + w2*EFD(A,B) • Properties: • Identity • Commutative • Positive semi-definite

  16. Activity Similarity • Two types of differences • core structural differences (csd) • event frequency differences (efd) • Properties: • Identity • Commutative • Positive semi-definite

  17. Activity Similarity • Two types of differences • core structural differences (csd) • event frequency differences (efd) • Properties: • Identity • Commutative • Positive semi-definite

  18. Activity Discovery and Anomalous Activity Explanation • Anomaly - “deviation” from the “common” or “regular” • Key Questions: • ‘representation’ of activities • ‘regular’ activities • ‘different’ from regular • ‘explain’an anomaly Activity similarity - Activity discovery

  19. Activity Sub-Class Discovery • Recall: regular activities occur frequently and are similar to each other • Activity Sub-Class Discovery - a Graphic Theoretic problem of finding maximal cliques in edge-weighted graphs • Maximal Cliques: overall similarity between clique nodes greater than some value, addition of any other node would reduce the overall clique similarity

  20. Activity Sub-Class Discovery • Recall: regular activities occur frequently and are similar to each other • Activity Sub-Class Discovery - a Graphic Theoretic problem of finding maximal cliques in edge-weighted graphs • Maximal Cliques: overall similarity between clique nodes greater than some value, addition of any other node would reduce the overall clique similarity

  21. Activity Sub-Class Discovery • Recall: regular activities occur frequently and are similar to each other • Activity Sub-Class Discovery - a Graphic Theoretic problem of finding maximal cliques in edge-weighted graphs • Maximal Cliques: overall similarity between clique nodes greater than some value, addition of any other node would reduce the overall clique similarity

  22. Activity Sub-Class Discovery • Sequentially find maximal cliques in edge weighted graph of activities • Activities different enough from all the regular activities are anomalies

  23. Activity Sub-Class Discovery • Sequentially find maximal cliques in edge weighted graph of activities • Activities different enough from all the regular activities are anomalies

  24. Activity Sub-Class Discovery • Sequentially find maximal cliques in edge weighted graph of activities • Activities different enough from all the regular activities are anomalies

  25. Activity Discovery and Anomalous Activity Explanation • Anomaly - “deviation” from the “common” or “regular” • Key Questions: • ‘representation’ of activities • ‘regular’ activities • ‘different’ from regular • ‘explain’an anomaly

  26. Activity Discovery and Anomalous Activity Explanation • Anomaly - “deviation” from the “common” or “regular” • Key Questions: • ‘representation’ of activities • ‘regular’ activities • ‘different’ from regular • ‘explain’an anomaly Activity classification - Anomaly detection

  27. Activity Classification • Compute weighted similarity between a new activity T and previous class members as: • Select membership sub-class as:

  28. Activity Classification • Compute weighted similarity between a new activity T and previous class members as: • Select membership sub-class as:

  29. Activity Discovery and Anomalous Activity Explanation • Anomaly - “deviation” from the “common” or “regular” • Key Questions: • ‘representation’ of activities • ‘regular’ activities • ‘different’ from regular • ‘explain’an anomaly Activity classification - Anomaly detection

  30. Anomaly Detection • Define function • Learn the detection threshold from training data

  31. Anomaly Detection • Define function • Represents the within-Class difference of the test activity w.r.t. previous class members • Pick a particular threshold to detect anomalies

  32. Anomaly Detection • Define function • Represents the within-Class difference of the test activity w.r.t. previous class members • Pick (learn) a particular threshold to detect anomalies

  33. Activity Discovery and Anomalous Activity Explanation • Anomaly - “deviation” from the “common” or “regular” • Key Questions: • ‘representation’ of activities • ‘regular’ activities • ‘different’ from regular • ‘explain’an anomaly

  34. Anomaly Explanation • Explanatory features: • Consistent • Frequent • Explanation based on features that were: • Deficientfrom an anomaly but were frequently and consistentlypresent in regular members • Extraneous in an anomaly but consistently absentfrom the regular members

  35. Anomaly Explanation • Explanatory features: • Consistent • Frequent • Explanation based on features that were: • Deficientfrom an anomaly but were frequently and consistentlypresent in regular members • Extraneous in an anomaly but consistently absentfrom the regular members

  36. Anomaly Explanation • Explanatory features: • Consistent • Frequent • Explanation based on features that were: • Deficientfrom an anomaly but were frequently and consistentlypresent in regular members • Extraneous in an anomaly but consistently absentfrom the regular members

  37. Anomaly Explanation

  38. Experimental Setup – Loading Dock • Barns & Nobel Loading Dock Area • One month worth of data: • 5 days a week – 9 a.m. till 5 p.m. • Event Vocabulary – 61 events • 195 activities: • 150 train activities + 45 test activities Bird’s Eye View of Experimental Setup

  39. Results General Characteristics of Discovered Activity Classes • UPS Delivery Vehicles • Fed Ex Delivery Vehicles • Delivery Trucks – multiple packages delivered • Cars and vans, only 1 or 2 packages delivered • Motorized cart used to pick and drop packages • Van deliveries – no use of motorized cart • Delivery trucks – multiple people

  40. Results General Characteristics of Discovered Activity Classes Few of the detected Anomalies • UPS Delivery Vehicles • Fed Ex Delivery Vehicles • Delivery Trucks – multiple packages delivered • Cars and vans, only 1 or 2 packages delivered • Motorized cart used to pick and drop packages • Van deliveries – no use of motorized cart • Delivery trucks – multiple people • Back door of delivery not closed • (b) More than usual number of people • involved in unloading • (c) Very few vocabulary events performed

  41. Results • Are the detected anomalous activities ‘interesting’ from human view-point? Anecdotal Validation: • Studied 7 users • Showed each user 8 regular activities selected randomly • Showed each user 10 test activities, 5 regular and 5 detected anomalous activities • 8 out of 10 activity-labels of the users matched the labels of our system • Probability of this match happening by chance is 4.4%

  42. Experimental Setup – House Environment • House environment – Commercially available strain gages • Five month worth of daily data (151 days): • Event Vocabulary – 16 events • 151 activities Top View of Experimental Setup

  43. Visualization of Discovered Clusteres 1 2 5 3 4

  44. Activity Discovery and Anomalous Activity Explanation - Recap • Anomaly - “deviation” from the “common” or “regular” • Key Questions: • ‘representation’ of activities • ‘regular’ activities • ‘different’ from regular • ‘explain’an anomaly

  45. Hard Question(s) • Importance of semantically meaningful activity-classes? • If not – can we construct a rules to translate computer-discovered classes to something human interpretable?