Review of probability
This presentation is the property of its rightful owner.
Sponsored Links
1 / 16

Review of Probability PowerPoint PPT Presentation


  • 66 Views
  • Uploaded on
  • Presentation posted in: General

Review of Probability. Definitions (1). Event space : pretty much any set of things, we’ll usually call it E if we have to refer to it. Random variable : a variable whose possible values are taken from an event space. Usually denoted with a capital letter.

Download Presentation

Review of Probability

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Review of probability

Review of Probability


Definitions 1

Definitions (1)

Event space: pretty much any set of things, we’ll usually call it E if we have to refer to it.

Random variable: a variable whose possible values are taken from an event space. Usually denoted with a capital letter.

Probability distribution: a function Pwith event space E is a probability distribution if it has these properties:

1.

2.

3.


Review of probability

Quiz

  • Let’s say I have a random variable X for a coin, with event space {H, T}.

    If the probability P(X=H) is 0.5, what is P(X=T)?

    2. If P(X=H)=0.25, what is P(X=T)?


Definitions 2

Definitions (2)

Joint distribution: A probability distribution of two or more random variables. The event space for this distribution is the cross product of the event space of the individual random variables.

E.g. Let X1 be a random variable for a coin flip. Let X2 be a random variable for a second coin flip.

P(X1, X2) is a joint distribution over all possible values for both coin flips.


Review of probability

Quiz

How many events are in the event space for flipping two coins?

Name two of these events.


Definitions 3

Definitions (3)

Marginal distribution: This is just any probability distribution, but people use it to refer to a distribution over one variable when they’ve separately introduced a joint distribution over that variable and a second variable.

E.g., if I have a joint distribution P(X1, X2), then P(X1) is a marginal distribution over X1, and P(X2) is a marginal distribution over X2.


Important equation

Important Equation

Marginalization rule, or rule of Total Probability:

This rule gives a way of figuring out a marginal distribution from a joint distribution.


Definitions 4

Definitions (4)

Conditional distribution: A conditional distribution over random variable X given random variable Y is written as P(X|Y=e), and is defined as:

Notice that in this conditional distribution, the probabilities for X need to sum to one.

Confusingly, people often say that P(X|Y) is a conditional distribution. However, this is actually a family of many different probability distributions, one for each value of Y.


Important equation 2

Important Equation (2)

Marginalization rule, or rule of Total Probability,

Second Version:

Really just the same as before, but written slightly differently.


Definitions 5

Definitions (5)

Independence: Random variables X and Y are independent (denoted ) if:

P(X, Y) = P(X)P(Y), for all possible events

Conditional Independence: Random variables X and Y are conditionally independent given random variable Z (denoted ) if:

P(X, Y|Z) = P(X|Z)P(Y|Z), for all possible events


Review of probability

Quiz

  • Suppose I flip a coin 3 times. Each time has P(H)=0.5. Assume the three coin tosses are independent. What is P(H, H, H)?

  • Suppose I flip the coin 4 times, and let the random variable for the i-th time be Xi. What is P(X1=X2=X3=X4)?

  • What is the probability that, in 4 coin flips, I get at least 3 heads?


Review of probability

Quiz

Suppose I have one normal coin (P(X1=H)=0.5), and one weird coin with the following properties:

P(X2=H|X1=H)=0.9

P(X2=T|X1=T)=0.8

If I flip X1 and then X2, what is P(X2=H)?


Important equations so far

Important Equations so far

Complementarity:

Conditional Probability:

Marginalization (or total probability):

or:

Common Mistake: The following looks like the complementarity rule, but there is no guarantee that it is true, and quite often it will not be true:

Very bad, don’t do this:


Quiz diabetes

Quiz: Diabetes

P(D)=0.01 (called the prior probability)

Test for diabetes is either + or –

P(+|D)=0.9

P(-|D)=0.8

P(-|D) =

P(+|D) =

P(+, D) =

P(-, D) =

P(+, D) =

P(-, D) =

P(D|+) =


Bayes rule

Bayes Rule

Likelihood

Prior

Marginal Likelihood

Bayes Rule:

or:

Proof of Bayes Rule:

  • (def. of cond. prob.)

  • (also def. of cond. prob.)

  • (multiply both sides of step 2 by P(Y))

  • (substitute LHS of step 3 into numerator of step 1)

Posterior


Bayes rule for diabetes test

Bayes Rule for Diabetes Test

P(D|+) = ?


  • Login