1 / 20

Special Topic on Image Retrieval

Special Topic on Image Retrieval. Local Feature Matching Verification. Geometric Verification. Motivation Remove false matches by checking geometric consistency. Red line : geometric consistent match Blue line : geometric inconsistent match. Global Verification: RANSAC.

chanda-kim
Download Presentation

Special Topic on Image Retrieval

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Special Topic on Image Retrieval Local Feature Matching Verification

  2. Geometric Verification • Motivation • Remove false matches by checking geometric consistency Red line: geometric consistent match Blue line: geometric inconsistent match

  3. Global Verification: RANSAC • Take RANSAC as an example • Check geometric consistency from matched feature pairs. Random sampling

  4. Local Geometric-Verification • Locally nearest neighbors (Video Goole, cvpr’03) • Matched regions should have a similar spatial layout. • For each match define its search area • Region in the search area that also matches casts a vote for the image • Reject matches with no support • Drawback • Sensitive to clutter

  5. Hamming Embedding (ECCV’08) • Introduced as an extension of BOV [Jegou 08] • Combination of • A partitioning technique (k-means) • A binary code that refine the descriptor • Representation of a descriptor x • Vector-quantized to q(x) as in standard BOV • short binary vector b(x) for an additional localization in the Voronoi cell • Two descriptors x and y match iif

  6. Hamming Embedding • Binary signature generation • Off-line learning • Random matrix generation • Descriptor projection and assignment • Median values of projected descriptors • On-line binarization • Quantization assignment • Descriptor projection • Computing the signature:

  7. Local Geometric-Verification • Bundled feature (CVPR’09) • Group local features in local MSER region. • Increase discriminative power of visual words. • Allowed to have large overlap error. • Bundle comparison: • Mm(q; p): number of common visual words between two bundles • Mg(q; p): inconsistency of geometric order in x- and y- direction. • Drawbacks: Infeasible for rotated bundles.

  8. Local Geometric-Verification Bundled feature (CVPR’09) • Visual words are bundled in MSER regions. • Spatial consistency for bundled features is utilized to weight visual words. # of shared visual words • Great performance for partial-dup detection in over 1 M database • Drawbacks: Infeasible for rotated bundles. Spatial consistency • Z. Wu, J. Sun, and Q. Ke, “Bundling Features for Large Scale Partial-Duplicate Web Image Search,” CVPR 09

  9. Global Verification: RANSAC • RANSAC: remove outliers by inlier classification • Inliers: true matched features • Outliers: false matched features • Assumption of RANdomSAmpleConsensus (RANSAC) • The original data consists of inliers and outliers. • A subset of inliers can estimate a model to optimally explain the inliers. • Estimate the affine transformation by RANSAC • Procedure: Iteratively select a random subset as hypothetical inliers • A model is fitted to the hypothetical inliers. • All other data are tested against the fitted model for inlier classification. • The model is re-estimated from all hypothetical inliers. • The model is evaluated by estimating the error of the inliers relative to the model. • Drawbacks: Computationally expensive, not scalable Fischler, et al., RANdomSAmpleConsensus: a paradigm for model fitting with applications to image analysis and automated cartography, Comm. of the ACM, 24:381-395, 1981

  10. Spatial Coding for Geometric Verification (ACM MM’ 10) • Motivation • Encode local features’ relative positions into compact binary maps • Check spatial consistency of local matches for geometric verification • Spatial coding maps • Relative spatial positions between local features. • Very efficient and high precision Zhou & Tian, Spatial Coding for large scale partial-duplicate image search. ACM Multimedia 2010.

  11. Spatial Map Generation • In previous case, each quadrant has one part • Consider each quadrant is uniformly divided into two parts. = Rotate 45 degree counterclockwise

  12. Spatial Map Generation • Generalized spatial map: GX and GY • Each quadrant is uniformly divided into r parts. X-map X-map X-map Y-map Y-map Y-map k=0 k=1 … … k=r-1

  13. Generalized Spatial Coding • Spatial coding maps: • Each quadrant uniformly divided into r parts. • Decompose the division into r sub-division. • Rotate each sub-division to align the axis. New featurelocations after rotation : Generalized spatial maps:

  14. Spatial Verification • Verification with spatial maps GX and GY • Compare the spatial maps of matched features: • k=0, …, r-1; i, j=1, …, N; N: number of matched features • Find and delete the most inconsistent matched pair, recursively: Vx: inconsistent degree in X-map Vy: inconsistent degree in Y-map Identify i* and remove

  15. 5 y 1 2 1 2 3 3 4 4 x 5

  16. Geometric Verification with Coding Maps 5 1 y 2 1 2 3 3 4 4 x 5 SUM

  17. 4 4 4 Image Plane Division (TOMCCAP’ 10) 3 3 3 2 2 2 1 1 1 5 5 5 4 4 4 (c) (a) (b) 3 3 3 2 2 2 5 5 5 1 1 1 (f) (d) (e)

  18. 4 Geometric Square Coding 3 2 • Coordinate adjustment • Square coding map 1 5 4 3 2 5 Generalized map: 1

  19. Geometric Fan Coding • Fan coding maps • Coordinate adjustment • Generalized coding maps

  20. Geometric Verification • Compare the fan coding maps of matched features: • Inconsistency measurement from geometric fan coding: • Inconsistency measurement from geometric square coding: • Inconsistency matrix:

More Related