Why probability?

1. Why probability? It’s cool Machine learning

2. A = {< 3} B = {even} class SampleSpace { List<SamplePoints> allSamplePoints; } (2) (4) (6) class SamplePoint { String description; List<Event> eventsIAmAMemberOf; float probability; } (3) (1) (5) C = {odd} class ModelOfExperiment { SamplePoint doExperiment(); SampleSpace getSampleSpace(); //can generate this using tree or //coordinate system } class Event { String description; List<SamplePoint> samplePointsIContain; float computeProbability();//sum over sample pts } class EventSpace { List<Event> specialSetOfEvents; } Conditional Probability P(A | B) = P(AB)/P(B) Event condProb(Event A, Event B) Independence P(A | B) = P(A), or P(AB) = P(A)P(B) boolean areIndependent(Event A, Event B) Conditional Independence P(AB | C) = P(A|C)P(B|C) boolean areCondInd(Event A, Event B, Event C) Bayes Theorem, Combinations, Permutations Event complement(Event e) Event intersection(Event A, Event B) Event union(Event A, Event B)

3. Random Variables Chapter 2

4. Random Variable Numerical attribute of an experimental outcome. Discrete Random Variable Continuous Random Variable

5. Example Experiment: Flip a coin 3 times. h = total # of heads r = length of longest run (eg. 2 tails in a row)

6. Relationship btw. random variables and events TTH HHT THT HTH TTT HTT THH HHH TTT HHH TTH HTT HHT THH THT HTH

7. Example: ph(0) = 1/8 ph(1) = 3/8 ph(2) = 3/8 ph(3) = 1/8 Probability Mass Function (PMF) For discrete random variables: PMF = ph(h0) = probability that the experimental outcome will have h = h0

8. Example (cont.) Graph of PMF

9. Compound Probability Mass Function (PMF) Example:

10. Conditional Probability P(x0|y0) = P(x0y0) / P(y0) Independence x, y are independent iff: For all x0, y0: P(x0y0) = P(x0)*P(y0) Conditional Independence x, y are conditionally independent iff: For all x0, y0: P(x0y0 | A) = P(x0|A)*P(y0|A)

11. Functions defined on random variables: A function on random variable(s) creates a new random variable: Examples: v = f(h) = h2 = {0, 1, 4, 9} w = f(h, r) = h*r = {0, 1, 2, 4, 9}

12. Expectation Weighted average of all possible outcomes. E[x] = ∑ [ x0 px (x0) ] E[g(x)] = ∑ [ g(x0) px(x0) ] E[w] = E[g(x,y)] = ∑ [ ∑ [ g(x0, y0) px,y(x0, y0) ]] Variance Measures the spread of the PMF around the expected value. σx2 = ∑ [ (x0 – E[x])2 p(x0) ] = E[ (x – E[x])2 ]

13. Continuous Sample Spaces, Event Spaces Experiments with infinitely many possible outcomes. Examples: Height, Weight… -1 -3 -2 0 1 2 3

14. Cumulative Density Function (CDF) Function px≤ (x0) such that:

15. Cumulative Density Function (CDF) Properties:

16. Probability Density Function (PDF) Function f(x) such that: event space

17. Unit-Impulse Function What is the derivative of f(x): 3 3 x0 What is ∫-∞ f ’(x) dx ?

18. PMF & CDF CDF PDF PDF

19. Compound Probability Density Function

20. Conditional Probability Conditional PDF

21. Independence Expectation