Human tracking techniques Niranjan kamat CSE 725

1. Human tracking techniquesNiranjan kamatCSE 725

2. Different Tracking Techniques • 1.Gaussian Mixture Model: • 1.Construct the model of the Background. • 2.Given sequence of background images find the mean and variance of mixture of the Gaussians. • 3.Given a new pixel find out the probability of the pixel value having come from the Gaussians.

3. 2.NCC:Normalized cross correlation • 1.For each pixel find out cross correlation with the background. • 2.Normalize by dividing the variances. • 3.Sometimes better results obtained by not normalizing.

4. 3.Entropy: • 1. The entropy is defined as the amount of information needed to exactly specify the state of the system, given what we know about the system. • 2. If the two images are exactly alike their entropy will be low. • 3. The lower the entropy the higher the likelihood that the pixel belongs to the background.

5. 4.Mutual Information: • 1. Mutual information of two random variables is a quantity that measures the mutual dependence of the two variables. • 2. The higher the mutual information the higher the likelihood of the pixel belonging to the background. • 3. Considered to be a stronger concept than entropy theoretically.

6. Gaussian Mixture Model • 1.Technique developed by Chris Stauffer and W.E.L. Grimson during CVPR 1999. • 2. The technique is so powerful and omnipresent in tracking,HCI,etc. that it won the Longuet-Higgins award at CVPR 2009.

7. Gaussian Mixture Model(continued) • We model the pixel history with K Gaussian Probability Density Functions. • The history of a pixel {x,y} is modeled as time series. • (vc.cs.nthu.edu.tw/.../Background_Estimation_with_Gaussian_Distribution_for_Image_Segmentation,_a_fast_approach.ppt)

8. Gaussian Mixture Model(continued) • The probability of observing the current pixel is given by • is the weight parameter that is used to describe by the Gaussian distribution • is a Gaussian distribution that has two parameters: is the mean of the Gaussian distribution at time t and is the covariance matrix at time instant t

9. Gaussian Mixture Model(continued) • A new pixel is said to match a distribution • If it is within 2.5 standard deviations from mean of the distribution • This distribution are updated with this pixel value

10. Gaussian Mixture Model(continued) is the learning rate defined by the user.I have set it as 0.1 which gives us good results.

11. Gaussian Mixture Model(continued) • The weight parameters of all distributions are updated as • is 1, for matched distribution 0, for unmatched distribution

12. Gaussian Mixture Model(continued) • If the current pixel didn’t match with any of the K distributions • The distribution with smallest weight is replaced by a new distribution • The new distribution with • The mean is set to the value of the current pixel • The variance is set large • The weight is set to a small value

13. Gaussian Mixture Model(continued) • Define which of distributions describing the history of a pixel result from background • 1. Order all distributions by a factor • 2. B first distributions are marked as background distribution • If a pixel is matched with one of these B distributions it is marked as a background pixel

14. Gaussian Mixture Model(continued) • 1. Remove the singular pixels and small blobs by eroding. • 2.Use connected components to segment out the persons.

15. Results • Here are the results to a normal video sequence. • As you can see the shadows have been removed so has the noise. • Further improved results can be obtained by edge detecting.

16. A Color Similarity Measure for Robust Shadow Removal in Real-Time • 1. This is a paper by Daniel Grest, Jan-Michael Frahm, and Reinhard Koch. • 2. Under normal conditions it gives us suitable results. • It is suitable for indoor use. However with the complexities of outside lighting, etc. it fails to live up to background modeling.

17. A Color Similarity Measure for Robust Shadow Removal in Real-Time • 1. To detect foreground regions we use a background reference image and calculate the difference image with the current one. • 2. To acquire a background reference a ground truth image is incorporated as a mean image of a sequence of the interaction area without the user. • 3. The segmentation of the foreground is often obtained by applying a threshold to the difference values.

18. A Color Similarity Measure for Robust Shadow Removal in Real-Time(Continued) • 1. Instead of calculating the variance for each pixel over time, we measure the variance of the pixel noise over the whole difference image each frame, leaving the segmented foreground region out.

19. A Color Similarity Measure for Robust Shadow Removal in Real-Time(Continued) • Let d(x,y) = r(x,y) - a(x,y) be the difference image value, r(x; y) the background reference image intensity and a(x,y) the current image intensity at image positions (x,y). The variance of noise in an image with size M X N is • Variance= Sum over the window ( d(i,j)-Mean Image)^2

20. A Color Similarity Measure for Robust Shadow Removal in Real-Time(Continued) • Assuming a Gaussian distribution of noise we set the segmentation threshold to 2 * Variance . Therefore 97.5% of noise are in the segmentation interval. • Each pixel belongs to the foreground, if • |d(x,y)-Mean Image| > 2 * Variance. • Nowwehavetoremovetheshadow.

21. A Color Similarity Measure for Robust Shadow Removal in Real-Time(Continued) • To measure the similarity of image parts it is possible to calculate the cross correlation over a given window size . Usually brightness invariance is desired, as given by the cross covariance, which is the average-free cross correlation.

22. A Color Similarity Measure for Robust Shadow Removal in Real-Time(Continued) • The cross covariance with window size M X N for two images a,b at image position (x,y) is then defined as • CC(a,b)(x,y) =(1/MN) Sum over window • (Pixel(x,y) of Image a-Mean window a) * (Pixel(x,y) of Image b-Mean window b) • To make it contrast invariant divide by the variance of each of the windows.

23. A Color Similarity Measure for Robust Shadow Removal in Real-Time(Continued) • For shadow pixels this correlation is near to one, so we eliminate the darker pixels in the segmented image, if the NCC of the pixel in the current image and the pixel in the reference image is greater than a threshold NCC. After removing pixels with the NCC thresholding, we apply a 3x3 closening operator to fill small holes in the contour.

24. A Color Similarity Measure for Robust Shadow Removal in Real-Time Results • Given below are the NCC results. • Using only NCC doesn’t take care of moving objects like trees blowing in wind,movingshadows,etc. • It is suitable only for indoors with restricted environment. • Using edge detection and NCC can give us improved results. • However, it is not as powerful as background modeling.

25. Entropy Results • If two images are similar their entropies should be close. • Hence, by taking the differences in entropy over a window, we can find the similarity. • This method doesn’t give us results as good as those for Background Modelling or even NCC.

26. Conclusion • Background Modelling gives us the nest results. • Done on a GPU it gives us real time speed.