1 / 21


Contents. Random variables, distributions, and probability density functions Discrete Random Variables Continuous Random Variables Expected Values and Moments Joint and Marginal Probability Means and variances Covariance matrices Univariate normal density

Download Presentation


An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.


Presentation Transcript

  1. Contents • Random variables, distributions, and probability density functions • Discrete Random Variables • Continuous Random Variables • Expected Values and Moments • Joint and Marginal Probability • Means and variances • Covariance matrices • Univariate normal density • Multivariate Normal densities 236607 Visual Recognition Tutorial

  2. Random variables, distributions, and probability density functions • Random variable X is a variable which value is set as a consequence of random events, that is the events, which results is impossible to know in advance. A set of all possible results is called a sampling domain and is denoted by . Such random variable can be treated as a “indeterministic” function X which relates every possible random event with some value . We will be dealing with real random variables • Probability distribution function is a function for which for every x 236607 Visual Recognition Tutorial

  3. Discrete Random Variable • Let X be a random variable (d.r.v.) that can assume m different values in the countable set • Let pibe the probability that X assumes the value vi: pi must satisfy: Mass function satisfy A connection between distribution and the mass function is given by 236607 Visual Recognition Tutorial

  4. Continuous Random Variable • The domain of continuous random variable (c.r.v.) is uncountable. The distribution function of c.r.v can be defined as where the function p(x) is called a probability density function . It is important to mention, that a numerical value of p(x) is not a “probability of x”. In the continuous case p(x)dx is a value which approximately equals to probability Pr[x<X<x+dx] 236607 Visual Recognition Tutorial

  5. Continuous Random Variable • Important features of the probability density function : 236607 Visual Recognition Tutorial

  6. Expected Values and Moments • The mean or expected value oraverage of x is defined by • If Y=g(X) we have: • The variance is defined as: where s is the standard deviation of x. 236607 Visual Recognition Tutorial

  7. Expected Values and Moments • Intuitively variance of x indicates distribution of its samples around its expected value (mean). Important property of the mean is its linearity: At the same time variance is not linear: • The k-th moment of r.v. X is E[Xk] (the expected value is a first moment). The k -th central moment is 236607 Visual Recognition Tutorial

  8. Joint and Marginal Probability • Let X and Y be 2 random variables with domains and For each pair of values we have a joint probability joint mass function • The marginal distributions for x and y are defined as • For c.r.v. marginal distributions can be calculated as 236607 Visual Recognition Tutorial

  9. Means and variances • The variables x and y are said to be statistically independent if and only if • The expected value of a function f(x,y) of two random variables x and y is defined as • The means and variances are: 236607 Visual Recognition Tutorial

  10. Covariance matrices • The covariance matrix S is defined as the square matrix whose ijth element sij is the covariance of xiand xj: 236607 Visual Recognition Tutorial

  11. Cauchy-Schwartz inequality From this we have the Cauchy-Schwartz inequality The correlation coefficient is normalized covariance It always . If the variables x and y are uncorrelated. If y=ax+b and a>0, then If a<0, then Question.Prove that if X and Y are independent r.v. then 236607 Visual Recognition Tutorial

  12. Covariance matrices • If the variables are statistically independent, the covariances are zero, and the covariance matrix is diagonal. • The covariance matrix is positive semi-definite: if w is any d-dimensional vector, then . This is equivalent to the requirement that none of the eigenvalues of S can ever be negative. 236607 Visual Recognition Tutorial

  13. Univariate normal density • The normal or Gaussian probability function is very important. In 1-dimension case, it is defined by probability density function • The normal density is described as a "bell-shaped curve", and it is completely determined by . • The probabilities obey 236607 Visual Recognition Tutorial

  14. Multivariate Normal densities • Suppose that each of the d random variables xi is normally distributed, each with its own mean and variance: • If these variables are independent, their joint density has the form • This can be written in a compact matrix form if we observe that for this case the covariance matrix is diagonal, i.e., 236607 Visual Recognition Tutorial

  15. Covariance matrices • and hence the inverse of the covariance matrix is easily written as 236607 Visual Recognition Tutorial

  16. Covariance matrices and • Finally, by noting that the determinant of S is just the product of the variances, we can write the joint density in the form • This is the general form of a multivariate normal density function, where the covariance matrix is no longer required to be diagonal. 236607 Visual Recognition Tutorial

  17. Covariance matrices • The natural measure of the distance from x to the mean m is provided by the quantity which is the square of the Mahalanobis distance from x to m. 236607 Visual Recognition Tutorial

  18. Example:Bivariate Normal Density where is a correlation coefficient; thus and after doing dot products in we get the expression for bivariate normal density: 236607 Visual Recognition Tutorial

  19. Some Geometric Features • The level curves of the 2D Gaussian are ellipses; the principal axes are in the direction of the eigenvectors of S, and the different width correspond to the corresponding eigenvalues. • For uncorrelated r.v. ( r=0 ) the axes are parallel to the coordinate axes. • For the extreme case of the ellipses collapse into straight lines (in fact there is only one independent r.v.). • Marginal and conditional densities are unidimensional normal. 236607 Visual Recognition Tutorial

  20. Some Geometric Features 236607 Visual Recognition Tutorial

  21. Law of Large Numbers and Central Limit Theorem • Law of large numbers Let X1, X2,…,be a series of i.i.d. (independent and identically distributed) random variables with E[Xi]=m. Then for Sn= X1+…+ Xn • Central Limit TheoremLet X1, X2,…,be a series of i.i.d. r.v. with E[Xi]=m and variance var(Xi)=s 2. Then for Sn= X1+…+ Xn 236607 Visual Recognition Tutorial

More Related