1 / 31

Structural Learning from Iconic Representations

Structural Learning from Iconic Representations. Herman M. Gomes* hmg@dsc.ufpb.br. Robert B. Fisher rbf@dai.ed.ac.uk. Institute of Perception, Action and Behaviour Division of Informatics Edinburgh University. * Supported by CNPq Brazil and DSC/COPIN/UFPB. Overview. Introduction

Download Presentation

Structural Learning from Iconic Representations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Structural Learning from Iconic Representations Herman M. Gomes* hmg@dsc.ufpb.br Robert B. Fisher rbf@dai.ed.ac.uk Institute of Perception, Action and Behaviour Division of Informatics Edinburgh University * Supported by CNPq Brazil and DSC/COPIN/UFPB.

  2. Overview • Introduction • Learning primitive object models • Learning model relationships • Case study • Conclusions and future work SBIA/IBERAMIA 2000, Atibaia SP

  3. Introduction • Traditional object recognition research • Geometric, symbolic or structure based recognition: CAD based 2D and 3D vision and 3D object recognition • Property, vector or feature based recognition: specific feature vectors, multiple filtering, global descriptors for shape, texture and colour, amongst others • Iconic or image based recognition: direct use of images, either complying with the traditional sensor architecture or using alternative representations • This work fits in the intersection of the above areas SBIA/IBERAMIA 2000, Atibaia SP

  4. Introduction • Question: Would it be possible to learn rigid geometric models from 2D image evidence (iconic object models) acquired from a sequence of scenes? SBIA/IBERAMIA 2000, Atibaia SP

  5. Introduction • Mechanisms from Biology • Foveated vision: retina-like image representation (log-polar) has useful properties • Visual attention: fixation gives insights where object features (or components) are likely to be found • Primal sketch: provides more compact representations for image data and cues for an attention mechanism SBIA/IBERAMIA 2000, Atibaia SP

  6. Introduction • System’s architecture Model base Attention Map Generic Scenes Update attention Primitive models Model relationships Feature planes Extract primal sketch planes Foveate Image Cluster objects SBIA/IBERAMIA 2000, Atibaia SP

  7. Introduction • Image representation • Gaussian receptive field function • Local contrast normalisation for estimating original reflectance information • Primal sketch features (edges, bars, blobs and ends) learned and extracted using a neural network approach • Log-polar SBIA/IBERAMIA 2000, Atibaia SP

  8. Introduction • Retinal and log-polar images SBIA/IBERAMIA 2000, Atibaia SP

  9. Introduction • Some of the extracted features SBIA/IBERAMIA 2000, Atibaia SP

  10. Learning primitive object models • Definition Primitive iconic model: set of regions or object instances that are similar to each other, organised into a geometric model and specified by means of the relative scales, orientations, positions and similarity scores for each pair of image regions SBIA/IBERAMIA 2000, Atibaia SP

  11. Learning primitive object models 5 1 11 12 4 6 9 2 3 7 8 10 13 Model X: {1,2,3,4,5,6,9,10,11} Model Y: {7,8,12,13} Y 7 8 12 13 Relative scales and orientations 7 (1.0, 0) (1.0, 180) (0.8, 270) (0.8, 90) 8 (1.0, 0) (0.8, 90) (0.8, 270) 12 (1.0, 0) (1.0, 180) 13 (1.0, 0) SBIA/IBERAMIA 2000, Atibaia SP

  12. Algorithm for each scene image S ido for each foveation point P ij on the scene do obtain the object feature f ij at the position P ij if the model base is empty then create a new model and store f ij on it else generate a set of scaled and rotated variations of f ij find the model Ft that gives the highest similarity score Cmax between its internal object features and one of the variations ifCmax > threshold store f ij in Ft for all f kl in Ftstore the similarity scores Sm(f ij, f kl) and the relative scales rS(f ij, f kl) and the relative orientations rO(f ij, f kl) elsecreate a new model and store f ij on it SBIA/IBERAMIA 2000, Atibaia SP

  13. Learning model relationships • Initial assumptions • The recognition of consistent geometric relations allows the inference of larger structured object models • At the moment we consider only 2D rigid body transformations SBIA/IBERAMIA 2000, Atibaia SP

  14. Learning model relationships • Overview of the solution • A graph is built from the output of the previous algorithm • Vertices represent instances of an image neighbourhood found in the scenes • Edges represent a relationships between two neighbourhoods • Intra and inter-model relationships are inferred by means of the cliques found in the graph SBIA/IBERAMIA 2000, Atibaia SP

  15. Learning model relationships • Vertices • How to distinguish between object feature instances of a same type that are structurally linked together and those that are not? • How to make the correspondence between sets of feature instances that appear at a same position in all objects? • We build hypothesis in the vertices by taking the full combinatorial set of object feature instances and applying an evaluation function that tells how good a hypothesis is SBIA/IBERAMIA 2000, Atibaia SP

  16. Learning model relationships • Vertices • Defined as the Cartesian product of the sets of object feature instances of a same model class found in each of the images in the sequence alternatively: • Missing features: a wild-card ‘*’ is added to f ibefore computing the Cartesian product SBIA/IBERAMIA 2000, Atibaia SP

  17. Learning model relationships • Vertices • Ranking: function of the similarity between its internal object instances • Pruning: allow only K *’s per vertex during the vertex creation process (K << N) • We define: Sm(*,*) = Sm(*,f j ) = Sm(f i ,*) = 1 SBIA/IBERAMIA 2000, Atibaia SP

  18. Learning model relationships • Edges • An edge e(a,b) relates two compatible vertices • Two vertices a and b are defined as compatible if: 1. For each pair of feature instances (a i, a j ) in a, which are related by a given scale and orientation R=(rS(a i, a j ), rO(a i, a j )), there exists another pair (b i, b j ) in bthat has internal components related through a similar relative scale and orientation; and 2. Each pair of feature instance co-ordinates (Pai, Pbi ) and (Paj, Pbj ) taken from the same vertex positions roughly define the same vector angle Aand length D,Q=(A, D), when taking into account the feature's relative scales and orientations. SBIA/IBERAMIA 2000, Atibaia SP

  19. Learning model relationships • Understanding edge creation Rija= (rS(a i, a j ), rO(a i, a j )) Rijb= (rS(b i, b j ), rO(b i, b j )) Qiab= A(Pai, Pbi ), D(Pai, Pbi )) Qjab= A(Paj, Pbj ), D(Paj, Pbj )) a=(a1, a2,..., aN) b=(b1, b2,..., bN) Rija ai aj bN Qiab Qjab ... ... ... bi bj aN Rijb Scene i Scene j Scene N SBIA/IBERAMIA 2000, Atibaia SP

  20. Learning model relationships • Edge ranking rS, rO ai aj bN A,D A,D ... ... ... bi bj aN rS, rO Scene i Scene j Scene N SBIA/IBERAMIA 2000, Atibaia SP

  21. Learning model relationships • Comparing relative scales rS(a i, a j ) ai aj bN ... ... ... bi bj aN rS(b i, b j ) Scene i Scene j Scene N SBIA/IBERAMIA 2000, Atibaia SP

  22. Learning model relationships • Comparing relative orientations rO(a i, a j ) ai aj bN ... ... ... bi bj aN rO(b i, b j ) Scene i Scene j Scene N SBIA/IBERAMIA 2000, Atibaia SP

  23. Learning model relationships • Comparing vector angles ai aj A(ai, bi ) bN A(a j, b j ) ... D(ai, bi ) ... ... D(a j, b j) bi bj aN Scene i Scene j Scene N SBIA/IBERAMIA 2000, Atibaia SP

  24. Learning model relationships • Comparing vector lengths SBIA/IBERAMIA 2000, Atibaia SP

  25. Learning model relationships • Edge pruning • Eliminate edges that link pairs of vertices with at least one common instance at the same position in the vertex list • Discard edges below a given threshold SBIA/IBERAMIA 2000, Atibaia SP

  26. Learning model relationships • Cliques • Sets of vertices that are maximally connected • A standard algorithm is used to find cliques in the graph G=(V,E) • Ranking: • Thresholding SBIA/IBERAMIA 2000, Atibaia SP

  27. Case study • Initial setup • Scenes created from synthetically generated variations on a same set of real input images • Interest points selected manually • Similarity function defined as the cross-correlation of grey level and primal sketch planes SBIA/IBERAMIA 2000, Atibaia SP

  28. Primitive object feature models found SBIA/IBERAMIA 2000, Atibaia SP

  29. Cliques found SBIA/IBERAMIA 2000, Atibaia SP

  30. Conclusions and future work • We provided some evidence that structured models can be learned in the context of an iconic vision system by using a graph-based representation and algorithm • The relative positions of object features are recorded and, by using the feature’s relative scales and orientations, possible relationships can be inferred SBIA/IBERAMIA 2000, Atibaia SP

  31. Conclusions and future work • There are still some issues that require further research: • Deciding for the best ranking functions and how they affect the final results • Learning primitive object models is computationally expensive • Vertex creation is exponential to the number of images • Designing more complex case studies is currently under development SBIA/IBERAMIA 2000, Atibaia SP

More Related