Formation et Analyse d’Images Session 10

1. Formation et Analyse d’ImagesSession 10 Daniela Hall 5 December 2005

2. Course Overview • Session 1 (19/09/05) • Overview • Human vision • Homogenous coordinates • Camera models • Session 2 (26/09/05) • Tensor notation • Image transformations • Homography computation • Session 3 (3/10/05) • Camera calibration • Reflection models • Color spaces • Session 4 (10/10/05) • Pixel based image analysis • 17/10/05 course is replaced by Modelisation surfacique

3. Course overview • Session 5 + 6 (24/10/05) 9:45 – 12:45 • Contrast description • Hough transform • Session 7 (7/11/05) • Kalman filter • Session 8 (14/11/05) • Tracking of regions, pixels, and lines • Session 9 (21/11/05) • Gaussian filter operators • Session 10 (5/12/05) • Scale Space • Session 11 (12/12/05) • Stereo vision • Epipolar geometry • Session 12 (16/01/06): exercises and questions

4. Session Overview • Scale space • Application: object recognition • Application: logo detection in sports videos

5. Scalability of Gaussian derivatives • Gaussian derivatives can be scaled due to the explicit scale parameter sigma.

6. Example: derivatives at several scales σ=1 σ=4 σ=2 σ=16 σ=8

7. Scale of image features • It would be interesting to know which are the interesting scales of an object. • Interesting scales: scales at which an important structure is visible. • In the example: the interesting scale (if we are interested in the people) is somewhere between 2.0 and 4.0. • Scale space theory answers these questions and provides a method to describe local features invariant to scale.

8. Scale of image features • Do these image features correspond?

9. Scale space representation • L is the scale space representation of I(x,y). • L is obtained by smoothing I with Gaussian kernel G(s). • G(s) is the natural choice for building up a scale space.

10. Scale Space • Scale space representation x s y I(x,y) x L(x,y,s)

11. Intrinsic feature scale [Lindeberg98] • Measurement procedure: • At a given spatial point (x,y) we sample the scale direction by applying a normalized derivative of different scales (scale signature f). • Different operators can be used for • Laplacian • Gradient norm

12. Gradient norm and Laplacian

13. Intrinsic feature scale [Lindeberg98] • Intrinsic scale: • The scale at which this normalized derivative assumes a maximum marks a feature containing interesting structure.

14. Scale signature of a feature

15. Intrinsic scale is stable over scales

16. Scale invariant feature description • Given a feature with intrinsic scale • The description components are normalized for

17. Scale invariant feature description • Compute intrinsic feature scale for each imagette. • Project imagette on scale normalised Gaussian derivatives. S(x,y,sk)=a0G(i,j,sk)+a1G’(i,j,sk)+a2G’’(i,j,sk)+... • Extract scale invariant signature (coefficients and intrinsic scale sk) v(S(x,y,sk))=(a0,a1,a2,...,sk) • Store all signatures with image id and position.

18. Scale invariant feature description • Do these image features correspond? Yes, because the signatures are close in feature space. • dist(v(S(x,y,σ)),v(S(sx,sy,sσ))) <eps S(x,y,σ) S(sx,sy,sσ)

19. Session Overview • Scale space • Application: object recognition • Application: logo detection in sports videos

20. Object recognition • Traning data: a set of images containing objects. • Test data: same objects in different configurations . • Goal: given a small number of measurements, recognize which objects are in the image.

21. Training data

22. Test data

23. Learning phase: sensing the world describing the measurements storage of measurement representation in appropriate data structure Recognition phase: sensing the world describing the measurements interpreting the measurements compare with learned measurements decide Recognition systems

24. Learning phase: sensing the world with a digital color camera discriminant feature description (texture & color) storage in indexing structure Recognition phase: sensing the world describing the measurements interpreting the measurements search possible matching candidates in indexing structure recognition by vote or hypothesis verification My recognition system

25. Choice of the feature space • The feature space should have few dimensions. • The description vector (set of Gaussian derivatives) should be chosen such that they capture the information necessary for recognition • Feature vector (v,θ,σ); • Global information (id of source image, pos within source image) • The description vector should be invariant to the transformations of the input data. • In case of viewpoint independent recognition, we require invariance to position, orientation, size, and affine transformations. • The choice are scale normalised Gaussian receptive fields oriented to the dominant direction of the imagette.

26. Feature description vector • describe local texture up to order 2 and local chrominance up to order 1 • description invariant to position, scale and orientation • similarity of features is proportional to distance in feature description space

27. Building the model Storage in appropriate indexing structure for efficient retrieval Select representative training images Scale invariant feature description

28. Recognition phase • Sensing the world • Describing the measurements • Interpreting the measurements • search possible matching candidates in the learned data • recognition by vote or hypothesis verification

29. Search matching candidates • Matching candidates are found by evaluating the distance in feature space. • All elements within a sphere with radius e from the query vector are matching candidates. • Tree structures allow efficient nearest neigbor search.

30. Recognize by vote • Distance evaluation produces a list of matching candidates. For each measurement we know from which object it came from (information stored during learning). • Every matching candidate votes for the object it came from. • The object with the higest number of votes is the winner (recognition result).

31. Experiment • Image indexing: 8 objects on uniform background. • Voting: • Select image features • Search matching candidates • Image for which votes the majority of the matching candidates gets a vote • Result: 99% recognition by votes after 9 different measurements

32. Open problems • Changes in viewpoint • Recognition under natural conditions (changes in lighting, viewing angle, …) • Capacity of tree structure is limited

33. Session Overview • Scale space • Application: object recognition • Application: logo detection in sports videos

34. Detecting Logos in Video • The video producer “sells” air-time of corporate publicity images (logos). • How can he measure the value of his product?

35. Detecting logos in video • Video publicity statistics • How many times does logo appear? • For each appearance • What was the duration? • How big was it? • How visible was it? • How near is it to the focus of attention? • Current estimates are obtained off-line by hand • This process is slow, costly and inaccurate.

36. The challenge of logo detection • Challenge: • Natural outdoor scenes • Unconstrained view angles • Rapid pan-tilt-zoom of cameras • Occlusions • Similar color/texture • Unknown target position, deformations • Goal: real time collection of logo statistics from sports videos

37. Model acquisition Learns model data for detection and identification • Data acquisiton: • Hand label example frames • Semi automatic labeling (label first frame, track following frames) • Learning phase: • color histograms for detection and tracking • Gaussian derivative histograms for identification • Problem: training data must capture operating conditions

38. Model acquisition

39. Detection by color histogram

40. Detection by histograms

41. Identification by Gaussian derivative histograms

42. Logo identification • Typical Example

43. Demo

44. Exercise • How can you count the number of flowers in the image and determine their scale?