Properties of Histograms and their Use for Recognition

1. Properties of Histograms and their Use for Recognition Stathis Hadjidemetriou, Michael Grossberg, Shree Nayar Department of Computer Science Columbia University New York, NY 10027

2. Motivation • Histogramming is a simple operation:

3. Motivation • Histograms have been used for: • Object recognition [Swain & Ballard 91, Stricker & Orengo 95] • Indexing from visual databases [Bach et al, 96, Niblack et al 93, Zhang et al 95] • Histogram advantages: • Efficient • Robust [Chatterjee, 96] • Histogram limitation: • Do not represent spatial information

4. Overview • Image transformations that preserve the histogram • Image structure through the multiresolution histogram • Multiresolution histogram compared with other features

5. Cut and rearrange regions Shuffle pixels Invariance of Histogram with Discontinuous Transformations

6. Rotation Shear Invariance of Histogram with Continuous Transformations

7. What is the complete class of continuous transformations that preserves the histogram?

8. Image: Map from continuous domain to intensities Model for Image Continuous domain

9. U U Histogram count for bin U ≡ Area bounded by level sets U Model for Histogram

10. Continuous Image Transformations • Vector fields, X, morph images [Spivak, 65]:

11. Original Gradient Transformations

12. Histograms of Gradient Transformations

13. Condition 1: Histogram Preservation and Local Area  …… ……  Histogram preservedLocal area preserved [Hadjidemetriou et al, 01]

14. Small region Local area preserveddivergence is zero [Arnold, 89] Condition 2: Local Area Preservation and Divergence • Divergence is rate of area change per unit area

15. Isovalue contours Hamiltonian flow Hamiltonian Fields • Fields along isovalue contours of an energy functionF • Flow of incompressible fluids[Arnold, 89]

16. Gradient of Hamiltonian of Computing Hamiltonian Fields • Compute gradient of F • Rotate gradient pointwise 900

17. Theorem Transformations preserve histogram of all images  corresponding field is Hamiltonian [Hadjidemetriou et al, CVPR, 00, Hadjidemetriou et al, IJCV, 01] Condition 3: Divergence and Hamiltonian Fields Divergence of field is zero Hamiltonian field[Arnold, 89]

18. Original Examples of Hamiltonian Transformations Linear:Translations, rotations, shears

19. Examples of Hamiltonian Transformations

20. h w 0 Border Preserving Hamiltonian Transformations

21. Examples of Windowed Hamiltonian Transformations

22. Identical histograms: Examples of Windowed Hamiltonian Transformations

23. Planar object tilt(f) causes shearing and scaling • Depth (z) causes scaling [Hadjidemetriou et al, 01] Weak Perspective Projection

24. The Hamiltonian transformations is the complete class of continuous image transformations that preserves the histogram

25. How can spatial information be embedded into the histogram?

26. Previous work on Features combining the Histogram with Spatial Information • Local statistics: • Local histograms [Hsu et al, 95, Smith & Chang, 96, Koenderink and Doorn, 99, Griffin, 97] • Intensity patterns [Haralick,79, Huang et al, 97] • One histogram: • Derivative filters[Schiele and Crowley, 00, Mel 97] • Gaussian filter[Lee and Dickinson, 94] • Many techniques are ad-hoc or not complete

27. G(ls2) Multiresolution Histogram

28. Limitations of Histograms Database of synthetic images with identical histograms [Hadjidemetriou et al, 01]

29. Matching with Multiresolution Histograms Match under Gaussian noise of st.dev. 15 graylevels:

30. Matching with Multiresolution Histograms Match under Gaussian noise of st.dev. 15 graylevels:

31. Image L h(L*G(l)) Multiresolution histogram Differences of histograms ? Image structure How is Image Structure Encoded in the Multiresolution Histogram?

32. Binj: ill-conditioned • Averages of bins: • where Pj are proportionality factors well-conditioned Histogram Change with Resolution and Spatial Information Spatial information

33. Averages = Generalized Fisher information measures of order q[Stam, 59, Plastino et al, 97] ≡ L is the image = D is the image domain Histogram Change with Resolution and Fisher Information Measures

34. P Fisher information measures (Analysis) Jq Image Structure Through Fisher Information Measures Image L h(L*G(l)) Multiresolution histogram Differences of histograms ? Image structure

35. Superquadrics: h=6.67 h=1.00 h=1.48 h=2.00 h=0.56 Histogram change with l is higher for complex boundary Shape Boundary and Multiresolution Histogram

36. Histogram change with l is proportional to number of texels (analytically) Texel Repetition and Multiresolution Histogram

37. Std. dev. of perturbation Histogram change with l decreases with randomness Texel Placement and Multiresolution Histogram

38. Cumulative histograms Differences of histograms between consecutive image resolutions Concatenate to form feature vector L1 norm Matching Algorithm for Multiresolution Histograms Burt-Adelson image pyramid

39. 5x5 44x44 89x89 179x179 Histogram Parameters • Bin width • Smoothing to avoid aliasing • Normalization: • Image size • Histogram size ……

40. Database of Synthetic Images 108 images with identical histograms [Hadjidemetriou et al, 01]

41. Sensitivity of Matching for Synthetic Images

42. 91 images with identical equalized histograms: 13 textures different rotations Database of Brodatz Textures

43. Match Results for Brodatz Textures Match under Gaussian noise of st.dev. 15 graylevels:

44. Sensitivity of Class Matching for Brodatz Textures

45. 8,046 images with identical equalized histograms: 61 materials under different illuminations [Dana et al, 99] Database of CUReT Textures

46. Match Results for CUReT Textures Match under Gaussian noise of st.dev. 15 graylevels:

47. Match Results for CUReT Textures Match under Gaussian noise of st.dev. 15 graylevels:

48. Sensitivity of Class Matching for CUReT Textures 100 randomly selected images per noise level

49. Embed spatial information into the histogram with the multiresolution histogram

50. How well does the multiresolution histogram perform compared to other image features?