Sketch-Based Shape Retrieval

1. Sketch-Based Shape Retrieval M. Eitz, R. Richter, K. Hildebrand, M. Alexa, TU Berlin; T. Boubekeur, Tele ParisTech – CNRS;

2. Outline • What is sketch based shape retrieval? • Sketch data base • Bag-of-features shape retrieval • GALIF: Gabor local line-based feature • Conclusions & Results

3. What is sketch based shape retrieval? • sketch 3D model

4. Sketch data base • Based on the Princeton Shape Benchmark (PSB), authors gather a lot of sketches. • Analysis result: users mostly sketch objects from a simple side or frontal view. • The sketches are free to download.

5. Sketch data base

6. Bag-of-features shape retrieval • Assuming there are two documents: • Bob likes to play basketball, Jim likes too • Bob also likes to play football games. • Construct a Dictionary: • Dictionary = {1:”Bob”, 2. “like”, 3. “to”, 4. “play”, 5. “basketball”, 6. “also”, 7. “football”, 8. “games”, 9. “Jim”, 10. “too”}

7. Bag-of-features shape retrieval • The two documents can be encoded by: • [1, 2, 1, 1, 1, 0, 0, 0, 1, 1] • [1, 1, 1, 1 ,0, 1, 1, 1, 0, 0] counts

8. Bag-of-features shape retrieval

9. Best-view selection • Uniformly distributed views: • Select d seeds on a unit sphere, • Lloyd relaxations iteratively, • d Voronoi cell centers as d view directions. • d ={22; 52; 102; 202}

10. Perceptually best views • Training set: manually define best and worst viewpoints in PSB • Learn a “best view classifier” from the training set using SVM. • Learn some best viewpoints based on the uniform viewpoints.

11. For each view direction vi , predict its probability pi = p(vi) of being a best view. • The probability is a smooth scalar field over a sphere and best views are local maxima.

12. GALIF: Gabor local line-based feature • Gabor filter : rotate an image by angle

13. Orientation-selective filter bank • Given kdifferent orientations, we can compute k different images: • (i)dft is the (inverse) discrete Fourier transformation • I --- input sketch • * --- point-wise multiplication

14. Local GALIF feature definition • I is divided into nxn regions • S, t <= n • i = 1, 2, ..., k. ------ orientataions

15. Conclusions & Results • Main differences with our paper: • Best view selection • Feature representation

16. Results

17. Q&A