Detecting Regions of Interest in Dynamic Scenes with Camera Motions

1. Detecting Regions of Interest in Dynamic Scenes with Camera Motions

2. OUTLINE • Introduction • Related Work • Detecting Regions of Interest • Evaluation and Results • Conclusion

3. Introduction • Automation of where to move the camera, which is driven by activities and events in the scene ,requires • knowledge of how the objects move in the scene • affordances related to how a camera can move to best capture these movements • ability to predict from motions of the camera and the dynamics of the scene where regions of interest are in the scene

4. Introduction

5. Introduction • Our contributions in this paper are: • A method to generate a stochastic motion field that represents a global motion tendency using Gaussian process regression (GPR). • Techniques for predicting important future locations from mean and variance fields computed from the stochastic vector field. • An evaluation method for measuring the goodness of predicted important regions.

6. Related Work • Kim et al [6] introduced an approach to measure global • tendencies from sparse set of motion with radial basis • function (RBF) interpolation. • This approach is used to predict the regions of • importance in the scene of the soccer videos captured • from multiple-static cameras

7. Detecting Regions of Interest

8. Detecting Regions of Interest

9. Covariance • 共變異數（Covariance）在機率論和統計學中用於衡量兩個變量的總體誤差。 • 如果兩個變數的變化趨勢一致，也就是說如果其中一個大於自身的期望值，另外一個也大於自身的期望值，那麼兩個變數之間的共變異數就是正值。 如果兩個變數的變化趨勢相反，那麼兩個變數之間的共變異數就是負值。

10. Covariance • Covariance Function ： • Specifies the covariance between pairs of random variables. • Covariance Matrix： • In probability theory and statistics, a covariance matrixis a matrix whose element in the i, j position is the covariance between the ith and jth elements of a random vector.

11. Regression

12. Gaussian Processes: Definition

13. Stochastic Motion Field • Let be a set of locations of extracted motion • Each location x has a set of noisy observed velocity vector • components: • yu(the velocity component in the u-axis), • yv(the velocity component in the v-axis), • (and optionally yt for modeling the component in the time-axis). • We assume that each velocity component at the location • follows the regression model • i.e., Normal distribution.

14. Gaussian Process Regression • We propose using the Gaussian process regression model, where • f(x) is a zero-mean Gaussian process with covariance function • It is completely specified by a mean function • (typically assumed to be 0) • and a covariance function

15. Gaussian Process Regression • If we have training data • The N x N covariance matrix K is now defined as • We then define the observation vector y = • y can be shown as a zero mean Gaussian process • with a covariance matrix • The posterior density for a test point is • a normal distribution with the mean y and the • variance var(y):

16. Gaussian Process Regression • We can then express the mean flow as a vector field for • two dimensional motions, • and for three dimensional motions as • with a variance for each velocity component • respectively.

17. Gaussian Process Regression

18. Detecting Locations of Convergence • Two problems. • Propagating every vector in the field has been shown to be computationally intensive. • extrapolated velocity vectors with large magnitudes can seriously bias accumulation and yield an unstable localization of converging points.

19. Detecting Locations of Convergence • Instead of propagating a magnitude of velocity, we transport certainty levels computed from GPR. • We transport the certainties only for the locations with high certainty levels. • Transporting the certainty level requires updating only the last destination point and is computationally more efficient.

20. Detecting Locations of Convergence • We first define an evaluating function • ,where is a motion field (mean field) • For example, if we denote the location of a motion as • the evaluating function iterates the locations by • ,where is computed from

21. Detecting Locations of Convergence

22. Measuring the Similarity with Field of View

23. Evaluation and Results

24. Evaluation and Results

25. Evaluation and Results

26. Evaluation and Results

27. Evaluation and Results

28. Evaluation and Results

29. Conclusion • We have shown that the prediction of the region of interests from • stochastic field using Gaussian Process Regression provides • robust results even with noisy motions from moving cameras. • In our future work, we will work on • Improving the scalability of the code-base acceleration since most computations consist of matrix-matrix products • Applying our approach for controlling actual robotic cameras in real-time.