Review of Probability. Definitions (1). Event space : pretty much any set of things, we’ll usually call it E if we have to refer to it. Random variable : a variable whose possible values are taken from an event space. Usually denoted with a capital letter.
Event space: pretty much any set of things, we’ll usually call it E if we have to refer to it.
Random variable: a variable whose possible values are taken from an event space. Usually denoted with a capital letter.
Probability distribution: a function Pwith event space E is a probability distribution if it has these properties:
If the probability P(X=H) is 0.5, what is P(X=T)?
2. If P(X=H)=0.25, what is P(X=T)?
Joint distribution: A probability distribution of two or more random variables. The event space for this distribution is the cross product of the event space of the individual random variables.
E.g. Let X1 be a random variable for a coin flip. Let X2 be a random variable for a second coin flip.
P(X1, X2) is a joint distribution over all possible values for both coin flips.
How many events are in the event space for flipping two coins?
Name two of these events.
Marginal distribution: This is just any probability distribution, but people use it to refer to a distribution over one variable when they’ve separately introduced a joint distribution over that variable and a second variable.
E.g., if I have a joint distribution P(X1, X2), then P(X1) is a marginal distribution over X1, and P(X2) is a marginal distribution over X2.
Marginalization rule, or rule of Total Probability:
This rule gives a way of figuring out a marginal distribution from a joint distribution.
Conditional distribution: A conditional distribution over random variable X given random variable Y is written as P(X|Y=e), and is defined as:
Notice that in this conditional distribution, the probabilities for X need to sum to one.
Confusingly, people often say that P(X|Y) is a conditional distribution. However, this is actually a family of many different probability distributions, one for each value of Y.
Marginalization rule, or rule of Total Probability,
Really just the same as before, but written slightly differently.
Independence: Random variables X and Y are independent (denoted ) if:
P(X, Y) = P(X)P(Y), for all possible events
Conditional Independence: Random variables X and Y are conditionally independent given random variable Z (denoted ) if:
P(X, Y|Z) = P(X|Z)P(Y|Z), for all possible events
Suppose I have one normal coin (P(X1=H)=0.5), and one weird coin with the following properties:
If I flip X1 and then X2, what is P(X2=H)?
Marginalization (or total probability):
Common Mistake: The following looks like the complementarity rule, but there is no guarantee that it is true, and quite often it will not be true:
Very bad, don’t do this:
P(D)=0.01 (called the prior probability)
Test for diabetes is either + or –
P(+, D) =
P(-, D) =
P(+, D) =
P(-, D) =
Proof of Bayes Rule:
P(D|+) = ?