- 77 Views
- Uploaded on
- Presentation posted in: General

Review of Probability

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Review of Probability

Event space: pretty much any set of things, we’ll usually call it E if we have to refer to it.

Random variable: a variable whose possible values are taken from an event space. Usually denoted with a capital letter.

Probability distribution: a function Pwith event space E is a probability distribution if it has these properties:

1.

2.

3.

- Let’s say I have a random variable X for a coin, with event space {H, T}.
If the probability P(X=H) is 0.5, what is P(X=T)?

2. If P(X=H)=0.25, what is P(X=T)?

Joint distribution: A probability distribution of two or more random variables. The event space for this distribution is the cross product of the event space of the individual random variables.

E.g. Let X1 be a random variable for a coin flip. Let X2 be a random variable for a second coin flip.

P(X1, X2) is a joint distribution over all possible values for both coin flips.

How many events are in the event space for flipping two coins?

Name two of these events.

Marginal distribution: This is just any probability distribution, but people use it to refer to a distribution over one variable when they’ve separately introduced a joint distribution over that variable and a second variable.

E.g., if I have a joint distribution P(X1, X2), then P(X1) is a marginal distribution over X1, and P(X2) is a marginal distribution over X2.

Marginalization rule, or rule of Total Probability:

This rule gives a way of figuring out a marginal distribution from a joint distribution.

Conditional distribution: A conditional distribution over random variable X given random variable Y is written as P(X|Y=e), and is defined as:

Notice that in this conditional distribution, the probabilities for X need to sum to one.

Confusingly, people often say that P(X|Y) is a conditional distribution. However, this is actually a family of many different probability distributions, one for each value of Y.

Marginalization rule, or rule of Total Probability,

Second Version:

Really just the same as before, but written slightly differently.

Independence: Random variables X and Y are independent (denoted ) if:

P(X, Y) = P(X)P(Y), for all possible events

Conditional Independence: Random variables X and Y are conditionally independent given random variable Z (denoted ) if:

P(X, Y|Z) = P(X|Z)P(Y|Z), for all possible events

- Suppose I flip a coin 3 times. Each time has P(H)=0.5. Assume the three coin tosses are independent. What is P(H, H, H)?
- Suppose I flip the coin 4 times, and let the random variable for the i-th time be Xi. What is P(X1=X2=X3=X4)?
- What is the probability that, in 4 coin flips, I get at least 3 heads?

Suppose I have one normal coin (P(X1=H)=0.5), and one weird coin with the following properties:

P(X2=H|X1=H)=0.9

P(X2=T|X1=T)=0.8

If I flip X1 and then X2, what is P(X2=H)?

Complementarity:

Conditional Probability:

Marginalization (or total probability):

or:

Common Mistake: The following looks like the complementarity rule, but there is no guarantee that it is true, and quite often it will not be true:

Very bad, don’t do this:

P(D)=0.01 (called the prior probability)

Test for diabetes is either + or –

P(+|D)=0.9

P(-|D)=0.8

P(-|D) =

P(+|D) =

P(+, D) =

P(-, D) =

P(+, D) =

P(-, D) =

P(D|+) =

Likelihood

Prior

Marginal Likelihood

Bayes Rule:

or:

Proof of Bayes Rule:

- (def. of cond. prob.)
- (also def. of cond. prob.)
- (multiply both sides of step 2 by P(Y))
- (substitute LHS of step 3 into numerator of step 1)

Posterior

P(D|+) = ?