Shadow Detection In Video

1. Shadow Detection In Video Submitted by: Hisham Abu saleh

2. Why detecting shadows? • Moving objects detection: • Shadows can cause object merging, object shape distortion or object loses. • Shadow points are detected as foreground • They differ from background. • They move the same as the object .

3. Why detecting shadows? cont.

4. Shadow Detection Algorithms • Algorithms are divided into: • Deterministic approaches • Model based • Best results. • Too Complex. • Non-Model based • Statistical approaches (use probabilistic functions). • Parametric methods. • Nonparametric methods. • Selecting parameters is critical issue.

5. Features presented by algorithms • Spectral • Gray level • Color (next lecture) • Spatial • Pixel level • Region level • Frame level • Temporal • Static • Dynamic

6. Algorithm Taxonomy

7. Algorithm Comparison • We will compare between four algorithms: • Statistical Nonparametric • Statistical Parametric • Deterministic Nonmodel-based (DNM1) • Deterministic Nonmodel-based (DNM2) • The deterministic model-based class has not been considered due to its complexity and due to its reliance on very specific task domain assumptions.

8. Statistical Nonparametric Approach • Horprasert • Color, Pixel level, Static. • Shadows have similar chromaticity, but lower brightness than the background model. • A statistical learning procedure is used to automatically determine the appropriate thresholds.

9. Statistical Parametric Approach • Mikie’ • Color, Region level, Dynamic. • The probability of a pixel belonging to shadow is computed by assuming the color of the pixel not shadowed, and by using an approximated linear transformation to estimate the color of the point covered by a shadow. • Manual segmentation of a certain number of frames has to be done to collect statistics and to compute the values of matrix of the transformation.

10. Deterministic Nonmodel-based Approach (DNM1) • Cucchiara, • Color, Pixel level, Static. • This algorithm works in the HSV color space (closer to the human perception of color and it has more accuracy in distinguishing shadows). • A shadow cast on a background does not change its hue significantly.

11. Deterministic Nonmodel-based Approach (DNM2) • Stauder • Gray level, Frame level, Dynamic. • The only algorithm that deals with penumbra. • Reference frame • Previous frame • Has limitations: Object speed, Noise sensetive. • Background frame

12. If illuminated If penumbra If umbra Deterministic Nonmodel-based Approach (DNM2) cont. • The image luminance: p(x,y) – The reflectance of the object surface. E(x,y) – irradiance. Describes the transition inside the penumbra Cp – Intensity of light source. CA – ambient light.

13. Deterministic Nonmodel-based Approach (DNM2) cont. • Assumptions: • 1. light source intensity cp is high. • The frame difference at a pixel changed by a moving cast shadow will be large. • If pixel is inside the umbra of a cast shadow at time instant k and outside the shadow at time instant k + 1. The reflectance of a static background does not change with time, thus the frame difference will be then:

14. Deterministic Nonmodel-based Approach (DNM2) cont. • 2. Camera and background are static. • 3. Background is plane, light source position is distant from background. • 4. Light source size and distance between moving object and background are not negligible compared to the distance between light source and object.

15. Deterministic Nonmodel-based Approach (DNM2) cont. • A. Detection of Static Background Edges • edges in the current and in the previous image are detected and classified into moving and static edges.

16. Deterministic Nonmodel-based Approach (DNM2) cont. • B. Detection of Uniform Changes of Shading • Frame Ratio: • If the frame ratio is locally spatially constant, a moving cast shadow is assumed at position x,y.

17. Deterministic Nonmodel-based Approach (DNM2) cont. • A shadow is decided to be in a region if it there is “darker” uniform region. • Static edges hint at a static background and can be exploited to detect nonmoving regions inside the frame difference.

18. Deterministic Nonmodel-based Approach (DNM2) cont. • C. Penumbra Detection • According to assumption 4 the cast shadow has penumbra. • The penumbra causes a soft luminance step at the contour of a shadow that is characterized by its step height, step width and its gradient.

19. Deterministic Nonmodel-based Approach (DNM2) cont. According to a table we put a candidate as a penumbra or not.

20. Algorithm comparison • Two important quality measures: • good detection • Low probability of misclassifying a shadow point. • Minimizing the shadow points classified as background/foreground. • good discrimination • the probability of classifying nonshadow points as shadow should be low.

21. Algorithm comparison • We define two metrics: TP – True Positive FN – False Negative S – Shadow F – Foreground (object)

22. Algorithm comparison • The Results: SNP – Very effective, sensitive to dark objects and noise. SP – Not very effective since the assumption that the transformation matrix is constant on the entire image. DNM1 – Good accuracy, stable performance. DNM2 – fails when the background is not planar.

23. Example

24. Physics-Based Shadow Detection • This step detects moving pixels (object, shadow, and some erroneous pixels).

25. Physics-Based Shadow Detection cont. • pixels on a detected surface cannot be shadow if they have higher intensity than the actual background. • This test does not reduce the shadow areas, but may successfully reduce object areas.

26. Physics-Based Shadow Detection cont.

27. Physics-Based Shadow Detection cont. • Shadow pixels falling on neutral surfaces, such as asphalt roads, tend to be more blue. • This test is not applied to all the pixels, but to neutral or gray surfaces that have low saturation (< 0.3)

28. Physics-Based Shadow Detection cont.

29. Physics-Based Shadow Detection cont. • This step performs surface segmentation. • The connectivity C of two neighboring pixels p1 and p2 with intensities u and v:

30. Physics-Based Shadow Detection cont. • H(u,v) is defined as follows: • If two neighboring pixels belong to the same surface they will have temporal ratios that are close together.

31. Physics-Based Shadow Detection cont.

32. Physics-Based Shadow Detection cont. • This step removes the effect of sky illumination. • We subtract the foreground pixel values from the background over the masked area.

33. Physics-Based Shadow Detection cont. • The user selects a patch of shadow region from a frame during the training. • The materials colors are stored to the verification step.

34. Physics-Based Shadow Detection cont. • This step performs matching of body color of various surfaces with the stored body color of materials that we expect to see in the scene. • The algorithm has training and testing phases.

35. Physics-Based Shadow Detection cont.

36. References • A. Prati, I. Mikic, M. Trivedi, R. Cucchiara, “Detecting Moving Shadows: Algorithms and Evaluation”, IEEE Transactions on pattern analysis and machine intelligence, vol. 25, no. 7, July 2003. • S. Nadimi, B. Bhanu, “Physical Models for Moving Shadow and Object Detection in Video”, IEEE Transactions on pattern analysis and machine intelligence, vol. 26, no. 8, August 2004. • J. Stauder, R. Mech, J. Ostermann, “Detection of Moving Cast Shadows for Object Segmentation”, IEEE Transactions on multi media, vol. 1, no. 1, March 1999.

37. References cont. • A. Prati, I. Mikic, C. Grana, M. Trivedi, “Shadow Detection Algorithms for Traffic Flow Analysis: a Comparative Study”.