1 / 42

ETISEO Annotation and Ground Truth Creation Rules for Video Object Tracking

This document outlines the ETISEO annotation rules, data structures, and the processes for creating ground truth data for video object tracking. It details the context, object definitions, event specifications, and how to handle occlusions and artifacts during annotation. Various data formats and tools, including Viper-GT, are explained. The document is essential for ensuring consistency and accuracy in video annotations, facilitating effective analysis and reference data generation.

lacy
Download Presentation

ETISEO Annotation and Ground Truth Creation Rules for Video Object Tracking

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ETISEO Annotation rules • Data structure • Annotation tool and format • Ground truth creation rules • Reference data generation

  2. Data structure - context • Context • Id • Type (ex: zone) • Name (ex: exit_zone) • Polygon

  3. Data structure - object • Physical object • Id • Start time • Type (ex: person, vehicle…) • Subtype (ex: car, van…) • Info2D • Info3D • Links • Speed • GT Nature • GT Status • GT Occluded (partially or completely) • GT Connected with • GT : specific to annotation purpose

  4. Data structure - event • Event • Id • Name • Start and end time • Status • Involved objects (physical object of interest and contextual object)

  5. ETISEO Annotation rules • Data structure • Annotation tool and format • Ground truth creation rules • Reference data generation

  6. Annotation tool and format • Annotation delivered with video: • Database - Excel file • Context annotation - Viper-GT tool and format • Annotation delivered at the end of Etiseo: • Ground truth - Viper-GT tool and format • Reference data (computed from GT) – Specific tool and Viper-GT format

  7. Annotation tool and format

  8. Annotation tool and format • Viper-GT format example • <?xml version="1.0" encoding="UTF-8"?> • <viper xmlns="http://lamp.cfar.umd.edu/viper#" xmlns:data="http://lamp.cfar.umd.edu/viperdata#"> • <config> <…> • </config> • <data> • <sourcefile filename="file:ETI-VS1-CR-4-C1.avi"> • <object framespan="1:100" id="4" name="physical_object"> • <attribute name="nature"> • <data:svalue value="physical_object_of_interest"/> • </attribute> • <attribute name="type"> • <data:svalue value="person"/> • </attribute> • <attribute name="info2D"> • <data:bbox framespan="1:1" height="182" width="86" x="138" y="59"/> • <data:bbox framespan="2:2" height="183" width="86" x="139" y="59"/> • <data:bbox framespan="3:3" height="185" width="86" x="141" y="60"/> • <data:bbox framespan="4:17" height="187" width="87" x="143" y="60"/> • <…> • <data:bbox framespan="40:100" height="212" width="10" x="3" y="10"/> • </attribute> • <…> • </sourcefile> • </data> • </viper>

  9. ETISEO Annotation rules • Data structure • Annotation tool and format • Ground truth creation rules • Reference data generation

  10. Ground truth creation rules • Rules for objects: • Contextual object • Static physical object of interest • Occlusion • Artefacts • Sudden illumination changes • Held object • Group - Crowd • Crowded video clip • Multi-view annotation • Rules for events

  11. Ground truth creation – Rules for objects • Contextual object • Annotated during its period of movement

  12. Ground truth creation rules • Rules for objects: • Contextual object • Static physical object of interest • Occlusion • Artefacts • Sudden illumination changes • Held object • Group - Crowd • Crowded video clip • Multi-view annotation • Rules for events

  13. Ground truth creation – Rules for objects • Static physical object of interest • Any physical object of interest is annotated on the whole video clip (ex: a parked vehicle) • First frame and last frame of a video clip

  14. Ground truth creation rules • Rules for objects: • Contextual object • Static physical object of interest • Occlusion • Artefacts • Sudden illumination changes • Held object • Group - Crowd • Crowded video clip • Multi-view annotation • Rules for events

  15. Ground truth creation – Rules for objects • Occlusions – general rules • Apparition of an object: the BBox increase with object apparition including only its visible part, • All BBox are limited by the image boundary, • Partial and complete occlusion will be differentiate in the evaluation (attribute “occluded”).

  16. Ground truth creation – Rules for objects • Static occlusions (by a contextual object) • Partial occlusion: full BBox is drawn including hidden part,

  17. Ground truth creation – Rules for objects • Static occlusions (by a contextual object) • Complete occlusion - two cases: • all “escapes” of the contextual object are visible: full BBox is drawn all the time,

  18. Ground truth creation – Rules for objects • all “escapes” of the contextual object are not visible: the object is no more annotated when completely occluded

  19. Ground truth creation – Rules for objects • Dynamic occlusion (by another object) • Partial or complete occlusion: in both cases, the full BBox is drawn including hidden part.

  20. Ground truth creation – Rules for objects • Split object due to occlusion • A unique BBox including all sub part is drawn.

  21. Ground truth creation rules • Rules for objects: • Contextual object • Static physical object of interest • Occlusion • Artefacts • Sudden illumination changes • Held object • Group - Crowd • Crowded video clip • Multi-view annotation • Rules for events

  22. Ground truth creation – Rules for objects • Artefacts • Only the part constituting the objects forms the BBox, artefacts (shadow, reflection…) are ignored for instant. • Annotated in video data set 2

  23. Ground truth creation rules • Rules for objects: • Contextual object • Static physical object of interest • Occlusion • Artefacts • Sudden illumination changes • Held object • Group - Crowd • Crowded video clip • Multi-view annotation • Rules for events

  24. Ground truth creation – Rules for objects • Sudden illumination changes (tagged frames) • Static object • Object in movement: position interpolation

  25. Ground truth creation rules • Rules for objects: • Contextual object • Static physical object of interest • Occlusion • Artefacts • Sudden illumination changes • Held object • Group - Crowd • Crowded video clip • Multi-view annotation • Rules for events

  26. Ground truth creation – Rules for objects • Held object • The holder and the held object are both annotated (if visible), • The carried object is connected to the holder. • Example: bag-person, person-bike, door-vehicle

  27. Ground truth creation – Rules for objects • Specific case • Person getting in/out a vehicle: annotated only when visible

  28. Ground truth creation rules • Rules for objects: • Contextual object • Static physical object of interest • Occlusion • Artefacts • Sudden illumination changes • Held object • Group - Crowd • Crowded video clip • Multi-view annotation • Rules for events

  29. Ground truth creation – Rules for objects • Group - Crowd • Group: multiple bounding boxes (persons + group) • Crowd : unique Bbox when individual annotation is impossible.

  30. Ground truth creation rules • Rules for objects: • Contextual object • Static physical object of interest • Occlusion • Artefacts • Sudden illumination changes • Held object • Group - Crowd • Crowded video clip • Multi-view annotation • Rules for events

  31. Ground truth creation – Rules for objects • Crowded video clip • Only object involved in event of interest are annotated. No false detections evaluation

  32. Ground truth creation rules • Rules for objects: • Contextual object • Static physical object of interest • Occlusion • Artefacts • Sudden illumination changes • Held object • Group - Crowd • Crowded video clip • Multi-view annotation • Rules for events

  33. Ground truth creation – Rules for objects • Multi-view annotation

  34. Ground truth creation rules • Rules for objects: • Contextual object • Static physical object of interest • Occlusion • Artefacts • Sudden illumination changes • Held object • Group - Crowd • Crowded video clip • Multi-view annotation • Rules for events

  35. Ground truth creation – Rules for events • Event – data structure • Id • Name • Start and end time • Status • Involved objects: • Id of the physical object of interest in GT files • Id of the contextual object in context information files

  36. Ground truth creation – Rules for events • Choice of event for each video clip in the event ontology => defined in the database xls file (provided with video)

  37. Ground truth creation – Rules for events • Automatic annotation of events with quantitative constraints, example: • Related to kinematics properties: stopped, running… • Related to spatial properties: close_to, inside_zone… • Manual annotation of other events: holding, pushing, waiting, gets_in/out,…

  38. Ground truth creation – Rules for events • Example

  39. ETISEO Annotation rules • Data structure • Annotation tool and format • Ground truth creation rules • Reference data generation

  40. Reference data generation • Filtering on object size • Object speed computation • Group generation • Multi-view and 3D reconstruction • Events with quantitative constraints generation

  41. ETISEO Annotation rules • Documentation: • ETI-Data structure and output format • ETI-Video Annotation Rules

  42. ETISEO Annotation rules • Questions …

More Related