1 / 21

AMI Tracking Evaluation

AMI Tracking Evaluation. Vítězslav Beran SERVITE, 13.1.2006 Department of Computer Graphic and Multimedia Faculty of Information Technology Brno University of Technology. Content. MULTI-OBJECT TRACKING EVALUATION Coverage test Configuration errors Identification errors

gwenb
Download Presentation

AMI Tracking Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. AMI Tracking Evaluation Vítězslav Beran SERVITE, 13.1.2006 Department of Computer Graphic and Multimedia Faculty of Information Technology Brno University of Technology

  2. Content • MULTI-OBJECT TRACKING EVALUATION • Coverage test • Configuration errors • Identification errors • AMI TRACKING EVALUATOR • User manual • Data formats • Implementation • Console Evaluator Vítězslav Beran, Ami Tracking Evaluator

  3. Multi-object Tracking Evaluation • Introduced in AMI WP4 Tracking: Evaluation scheme 1.0, 2006 • Two data sets • Ground Truth (GT) • Estimates (E) Vítězslav Beran, Ami Tracking Evaluator

  4. Coverage test • Two measurements • Recall • Precision • F-measure Vítězslav Beran, Ami Tracking Evaluator

  5. Configuration errors • Frame errors • False positive (FP) • False negative (FN) • Multiple trackers (MT) • Multiple objects (MO) • Configuration distance (CD) Vítězslav Beran, Ami Tracking Evaluator

  6. GTj 1 2 3 Ei 1 0,89 0,00 0,00 1 2 0,00 0,00 0,00 0 3 0,00 0,51 0,00 1 4 0,00 0,47 0,47 2 1 2 2 Configuration errors (cont.) • tI is coverage threshold Vítězslav Beran, Ami Tracking Evaluator

  7. Configuration errors (cont.) • Configuration Map • Configuration Errors Vítězslav Beran, Ami Tracking Evaluator

  8. Configuration distance • Relevant only together with other errors. Vítězslav Beran, Ami Tracking Evaluator

  9. Occlusion • t0 is occlusion threshold • when occlusion is set no error is computed Vítězslav Beran, Ami Tracking Evaluator

  10. Normalization • for easy comparison of tracking algorithm errors Vítězslav Beran, Ami Tracking Evaluator

  11. Identification errors • Sequence errors • Falsely identified tracker (FIT) • Falsely identified object (FIO) Vítězslav Beran, Ami Tracking Evaluator

  12. Identification errors (cont.) • Object purity (OP) • Track purity (TP) • Not interesting for our purpose. Vítězslav Beran, Ami Tracking Evaluator

  13. Example sequence Vítězslav Beran, Ami Tracking Evaluator

  14. Example sequence results Vítězslav Beran, Ami Tracking Evaluator

  15. AMI Tracking Evaluator • Windows platform application • tracking video dataevaluation • load / save data sets from file source • GT and E objects visualization • errors visualization • console version Vítězslav Beran, Ami Tracking Evaluator

  16. User interface Vítězslav Beran, Ami Tracking Evaluator

  17. Data formats - XML <!ELEMENT AVEvents (EventGroups?, EventTypes?, File?, Editor?)> ... <!ELEMENT File (Source*, TimeFormat?, Event*, Title*)> <!ELEMENT Source (#PCDATA)> <!ELEMENT TimeFormat (#PCDATA)> <!ELEMENT Event (ID, Time, Text?, Parameters?)> <!ELEMENT ID (#PCDATA)> <!ELEMENT Time (#PCDATA)> <!ELEMENT Text (#PCDATA)> <!ELEMENT Parameters EMPTY> <!ELEMENT Title (Time, Text)> ... <!ELEMENT Editor ANY> Vítězslav Beran, Ami Tracking Evaluator

  18. Data formats - TXT • Three format supported: • format 1 frame frameID object objectID BoxCenterX BoxCenterY Width/2 Height/2 object ... • format 2 image[frameID].* objectID minXPos minYPos maxXPos maxYPos • format 3 frameID objectID visibility minXPos minYPos maxXPos maxYPos Vítězslav Beran, Ami Tracking Evaluator

  19. Conclusion • tool for multi-object tracking evaluation • same tool by IDIAP is much more strict to data source formats • presentation, documentation and downloads available onhttp://www.fit.vutbr.cz/research/grants/ami/evaluator.html Vítězslav Beran, Ami Tracking Evaluator

  20. Future work • evaluate our evaluation tool Vítězslav Beran, Ami Tracking Evaluator

  21. Thank you for your attention. Any questions, please?

More Related