Review of probability
1 / 16

Review of Probability - PowerPoint PPT Presentation

  • Uploaded on

Review of Probability. Definitions (1). Event space : pretty much any set of things, we’ll usually call it E if we have to refer to it. Random variable : a variable whose possible values are taken from an event space. Usually denoted with a capital letter.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Review of Probability' - jude

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Definitions 1
Definitions (1)

Event space: pretty much any set of things, we’ll usually call it E if we have to refer to it.

Random variable: a variable whose possible values are taken from an event space. Usually denoted with a capital letter.

Probability distribution: a function Pwith event space E is a probability distribution if it has these properties:





  • Let’s say I have a random variable X for a coin, with event space {H, T}.

    If the probability P(X=H) is 0.5, what is P(X=T)?

    2. If P(X=H)=0.25, what is P(X=T)?

Definitions 2
Definitions (2)

Joint distribution: A probability distribution of two or more random variables. The event space for this distribution is the cross product of the event space of the individual random variables.

E.g. Let X1 be a random variable for a coin flip. Let X2 be a random variable for a second coin flip.

P(X1, X2) is a joint distribution over all possible values for both coin flips.


How many events are in the event space for flipping two coins?

Name two of these events.

Definitions 3
Definitions (3)

Marginal distribution: This is just any probability distribution, but people use it to refer to a distribution over one variable when they’ve separately introduced a joint distribution over that variable and a second variable.

E.g., if I have a joint distribution P(X1, X2), then P(X1) is a marginal distribution over X1, and P(X2) is a marginal distribution over X2.

Important equation
Important Equation

Marginalization rule, or rule of Total Probability:

This rule gives a way of figuring out a marginal distribution from a joint distribution.

Definitions 4
Definitions (4)

Conditional distribution: A conditional distribution over random variable X given random variable Y is written as P(X|Y=e), and is defined as:

Notice that in this conditional distribution, the probabilities for X need to sum to one.

Confusingly, people often say that P(X|Y) is a conditional distribution. However, this is actually a family of many different probability distributions, one for each value of Y.

Important equation 2
Important Equation (2)

Marginalization rule, or rule of Total Probability,

Second Version:

Really just the same as before, but written slightly differently.

Definitions 5
Definitions (5)

Independence: Random variables X and Y are independent (denoted ) if:

P(X, Y) = P(X)P(Y), for all possible events

Conditional Independence: Random variables X and Y are conditionally independent given random variable Z (denoted ) if:

P(X, Y|Z) = P(X|Z)P(Y|Z), for all possible events


  • Suppose I flip a coin 3 times. Each time has P(H)=0.5. Assume the three coin tosses are independent. What is P(H, H, H)?

  • Suppose I flip the coin 4 times, and let the random variable for the i-th time be Xi. What is P(X1=X2=X3=X4)?

  • What is the probability that, in 4 coin flips, I get at least 3 heads?


Suppose I have one normal coin (P(X1=H)=0.5), and one weird coin with the following properties:



If I flip X1 and then X2, what is P(X2=H)?

Important equations so far
Important Equations so far


Conditional Probability:

Marginalization (or total probability):


Common Mistake: The following looks like the complementarity rule, but there is no guarantee that it is true, and quite often it will not be true:

Very bad, don’t do this:

Quiz diabetes
Quiz: Diabetes

P(D)=0.01 (called the prior probability)

Test for diabetes is either + or –



P(-|D) =

P(+|D) =

P(+, D) =

P(-, D) =

P(+, D) =

P(-, D) =

P(D|+) =

Bayes rule
Bayes Rule



Marginal Likelihood

Bayes Rule:


Proof of Bayes Rule:

  • (def. of cond. prob.)

  • (also def. of cond. prob.)

  • (multiply both sides of step 2 by P(Y))

  • (substitute LHS of step 3 into numerator of step 1)