# Raffay Hamid, Amos Johnson, Samir Batta, Aaron Bobick, Charles Isbell, Graham Coleman - PowerPoint PPT Presentation

Raffay Hamid, Amos Johnson, Samir Batta, Aaron Bobick, Charles Isbell, Graham Coleman

1 / 47
Raffay Hamid, Amos Johnson, Samir Batta, Aaron Bobick, Charles Isbell, Graham Coleman

## Raffay Hamid, Amos Johnson, Samir Batta, Aaron Bobick, Charles Isbell, Graham Coleman

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
##### Presentation Transcript

1. Activity Discovery and Anomalous Activity Explanation Raffay Hamid, Amos Johnson, Samir Batta, Aaron Bobick, Charles Isbell, Graham Coleman

2. Activity Discovery and Anomalous Activity Explanation

3. Activity Discovery and Anomalous Activity Explanation • Anomaly - “deviation” from the “common” or “regular”

4. Activity Discovery and Anomalous Activity Explanation • Anomaly - “deviation” from the “common” or “regular” • Key Questions: • ‘representation’ of activities • ‘regular’ activities • ‘different’ from regular • ‘explain’an anomaly

5. Activity Discovery and Anomalous Activity Explanation • Anomaly - “deviation” from the “common” or “regular” • Key Questions: • ‘representation’ of activities • ‘regular’ activities • ‘different’ from regular • ‘explain’an anomaly

6. Activity Representation • Previous representations include: • Stochastic Context Free Grammars • Expectation Grammars • …..

7. Activity Representation • Previous representations include: • Stochastic Context Free Grammars • Expectation Grammars • ….. • Require some a priori information about activity structure

8. Activity Representation • Two pieces of information: • content • structure • Drawing from Natural Language Processing – treating documents as bags of words • Treat Activities as bags of event n-grams • Extraction of global structural information using local event statistics

9. Activity Representation • Two pieces of information: • content • structure • Drawing from Natural Language Processing – treating documents as bags of words –captures content well • Treat Activities as bags of event n-grams • Extraction of global structural information using local event statistics

10. Activity Representation • Two pieces of information: • content • structure • Drawing from Natural Language Processing – treating documents as bags of words –captures content well • Treat Activities as bags of event n-grams –captures activity structure • Extraction of global structural information using local event statistics

11. Activity Representation • Two pieces of information: • content • structure • Drawing from Natural Language Processing – treating documents as bags of words –captures content well • Treat Activities as bags of event n-grams –captures activity structure • Extraction of global structural information using local event statistics

12. Activity Discovery and Anomalous Activity Explanation • Anomaly - “deviation” from the “common” or “regular” • Key Questions: • ‘representation’ of activities • ‘regular’ activities • ‘different’ from regular • ‘explain’an anomaly

13. Activity Discovery and Anomalous Activity Explanation • Anomaly - “deviation” from the “common” or “regular” • Key Questions: • ‘representation’ of activities • ‘regular’ activities • ‘different’ from regular • ‘explain’an anomaly - Occur frequently - Are similar to each other

14. Activity Discovery and Anomalous Activity Explanation • Anomaly - “deviation” from the “common” or “regular” • Key Questions: • ‘representation’ of activities • ‘regular’ activities • ‘different’ from regular • ‘explain’an anomaly - Activity similarity - Activity discovery

15. Activity Similarity • Two types of differences • core structural differences (csd) • event frequency differences (efd) • Sim (A,B) = w1*CSD(A,B) + w2*EFD(A,B) • Properties: • Identity • Commutative • Positive semi-definite

16. Activity Similarity • Two types of differences • core structural differences (csd) • event frequency differences (efd) • Properties: • Identity • Commutative • Positive semi-definite

17. Activity Similarity • Two types of differences • core structural differences (csd) • event frequency differences (efd) • Properties: • Identity • Commutative • Positive semi-definite

18. Activity Discovery and Anomalous Activity Explanation • Anomaly - “deviation” from the “common” or “regular” • Key Questions: • ‘representation’ of activities • ‘regular’ activities • ‘different’ from regular • ‘explain’an anomaly Activity similarity - Activity discovery

19. Activity Sub-Class Discovery • Recall: regular activities occur frequently and are similar to each other • Activity Sub-Class Discovery - a Graphic Theoretic problem of finding maximal cliques in edge-weighted graphs • Maximal Cliques: overall similarity between clique nodes greater than some value, addition of any other node would reduce the overall clique similarity

20. Activity Sub-Class Discovery • Recall: regular activities occur frequently and are similar to each other • Activity Sub-Class Discovery - a Graphic Theoretic problem of finding maximal cliques in edge-weighted graphs • Maximal Cliques: overall similarity between clique nodes greater than some value, addition of any other node would reduce the overall clique similarity

21. Activity Sub-Class Discovery • Recall: regular activities occur frequently and are similar to each other • Activity Sub-Class Discovery - a Graphic Theoretic problem of finding maximal cliques in edge-weighted graphs • Maximal Cliques: overall similarity between clique nodes greater than some value, addition of any other node would reduce the overall clique similarity

22. Activity Sub-Class Discovery • Sequentially find maximal cliques in edge weighted graph of activities • Activities different enough from all the regular activities are anomalies

23. Activity Sub-Class Discovery • Sequentially find maximal cliques in edge weighted graph of activities • Activities different enough from all the regular activities are anomalies

24. Activity Sub-Class Discovery • Sequentially find maximal cliques in edge weighted graph of activities • Activities different enough from all the regular activities are anomalies

25. Activity Discovery and Anomalous Activity Explanation • Anomaly - “deviation” from the “common” or “regular” • Key Questions: • ‘representation’ of activities • ‘regular’ activities • ‘different’ from regular • ‘explain’an anomaly

26. Activity Discovery and Anomalous Activity Explanation • Anomaly - “deviation” from the “common” or “regular” • Key Questions: • ‘representation’ of activities • ‘regular’ activities • ‘different’ from regular • ‘explain’an anomaly Activity classification - Anomaly detection

27. Activity Classification • Compute weighted similarity between a new activity T and previous class members as: • Select membership sub-class as:

28. Activity Classification • Compute weighted similarity between a new activity T and previous class members as: • Select membership sub-class as:

29. Activity Discovery and Anomalous Activity Explanation • Anomaly - “deviation” from the “common” or “regular” • Key Questions: • ‘representation’ of activities • ‘regular’ activities • ‘different’ from regular • ‘explain’an anomaly Activity classification - Anomaly detection

30. Anomaly Detection • Define function • Learn the detection threshold from training data

31. Anomaly Detection • Define function • Represents the within-Class difference of the test activity w.r.t. previous class members • Pick a particular threshold to detect anomalies

32. Anomaly Detection • Define function • Represents the within-Class difference of the test activity w.r.t. previous class members • Pick (learn) a particular threshold to detect anomalies

33. Activity Discovery and Anomalous Activity Explanation • Anomaly - “deviation” from the “common” or “regular” • Key Questions: • ‘representation’ of activities • ‘regular’ activities • ‘different’ from regular • ‘explain’an anomaly

34. Anomaly Explanation • Explanatory features: • Consistent • Frequent • Explanation based on features that were: • Deficientfrom an anomaly but were frequently and consistentlypresent in regular members • Extraneous in an anomaly but consistently absentfrom the regular members

35. Anomaly Explanation • Explanatory features: • Consistent • Frequent • Explanation based on features that were: • Deficientfrom an anomaly but were frequently and consistentlypresent in regular members • Extraneous in an anomaly but consistently absentfrom the regular members

36. Anomaly Explanation • Explanatory features: • Consistent • Frequent • Explanation based on features that were: • Deficientfrom an anomaly but were frequently and consistentlypresent in regular members • Extraneous in an anomaly but consistently absentfrom the regular members

37. Anomaly Explanation

38. Experimental Setup – Loading Dock • Barns & Nobel Loading Dock Area • One month worth of data: • 5 days a week – 9 a.m. till 5 p.m. • Event Vocabulary – 61 events • 195 activities: • 150 train activities + 45 test activities Bird’s Eye View of Experimental Setup

39. Results General Characteristics of Discovered Activity Classes • UPS Delivery Vehicles • Fed Ex Delivery Vehicles • Delivery Trucks – multiple packages delivered • Cars and vans, only 1 or 2 packages delivered • Motorized cart used to pick and drop packages • Van deliveries – no use of motorized cart • Delivery trucks – multiple people

40. Results General Characteristics of Discovered Activity Classes Few of the detected Anomalies • UPS Delivery Vehicles • Fed Ex Delivery Vehicles • Delivery Trucks – multiple packages delivered • Cars and vans, only 1 or 2 packages delivered • Motorized cart used to pick and drop packages • Van deliveries – no use of motorized cart • Delivery trucks – multiple people • Back door of delivery not closed • (b) More than usual number of people • involved in unloading • (c) Very few vocabulary events performed

41. Results • Are the detected anomalous activities ‘interesting’ from human view-point? Anecdotal Validation: • Studied 7 users • Showed each user 8 regular activities selected randomly • Showed each user 10 test activities, 5 regular and 5 detected anomalous activities • 8 out of 10 activity-labels of the users matched the labels of our system • Probability of this match happening by chance is 4.4%

42. Experimental Setup – House Environment • House environment – Commercially available strain gages • Five month worth of daily data (151 days): • Event Vocabulary – 16 events • 151 activities Top View of Experimental Setup

43. Activity Discovery and Anomalous Activity Explanation - Recap • Anomaly - “deviation” from the “common” or “regular” • Key Questions: • ‘representation’ of activities • ‘regular’ activities • ‘different’ from regular • ‘explain’an anomaly

44. Hard Question(s) • Importance of semantically meaningful activity-classes? • If not – can we construct a rules to translate computer-discovered classes to something human interpretable?