Probability Theory Overview and Analysis of Randomized Algorithms

1 / 50

Probability Theory Overview and Analysis of Randomized Algorithms - PowerPoint PPT Presentation

Probability Theory Overview and Analysis of Randomized Algorithms. Analysis of Algorithms. Prepared by John Reif, Ph.D. Probability Theory Topics . Random Variables: Binomial and Geometric Useful Probabilistic Bounds and Inequalities. Readings. Main Reading Selections:

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

PowerPoint Slideshow about 'Probability Theory Overview and Analysis of Randomized Algorithms' - naava

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Probability Theory Overview and Analysis of Randomized Algorithms

Analysis of Algorithms

Prepared by

John Reif, Ph.D.

Probability Theory Topics
• Random Variables: Binomial and Geometric
• Useful Probabilistic Bounds and Inequalities
• Main Reading Selections:
• CLR, Chapter 5 and Appendix C
Probability Measures
• A probability measure (Prob) is a mapping from a set of events to the reals such that:
• For any event A

0  Prob(A)  1

• Prob (all possible events) = 1
• If A, B are mutually exclusive events, then

Prob(A  B) = Prob (A) + Prob (B)

Bayes’ Theorem
• If A1, …, An are mutually exclusive and contain all events then
Random Variable A (cont’d)
• Prob Distribution Function
Random Variable A (cont’d)
• If for Random Variables A,B
• Then “A upper bounds B” and “B lower bounds A”
Expectation of Random Variable A
• Ā is also called “average of A” and “mean of A” = A
Pair-Wise Independent Random Variables
• A,B independent if

Prob(A  B) = Prob(A) X Prob(B)

• Equivalent definition of independence
Bounding Numbers of Permutations
• n! = n x (n-1) x 2 x 1= number of permutations of n objects
• Stirling’s formula

n! = f(n) x (1+o(1)), where

Bounding Numbers of Permutations (cont’d)
• Note
• Tighter bound

= number of permutations of n objects taken k at a time

Bounding Numbers of Combinations

= number of (unordered) combinations of n objects taken k at a time

• Bounds (due to Erdos & Spencer, p. 18)
Bernoulli Variable
• Ai is 1 with prob P and 0 with prob 1-P
• Binomial Variable
• B is sum of n independent Bernoulli variables Ai each with some probability p
Poisson Trial
• Ai is 1 with prob Piand 0 with prob 1-Pi
• Suppose B’ is the sum of n independent Poisson trialsAi with probability Pi for i > 1, …, n
Hoeffding’s Theorem
• B’ is upper bounded by a Binomial Variable B
• Parameters n,p where

procedure

GEOMETRIC parameter p

¬

begin

V 0

loop with probability 1-p

goto

exit

Geometric Variable V
• parameter p
Probabilistic Inequalities
• For Random Variable A
Markov and Chebychev Probabilistic Inequalities
• Markov Inequality (uses only mean)
• Chebychev Inequality (uses mean and variance)
• If B is a Binomial with parameters n,p
Normal Distribution
• Bounds x > 0 (Feller, p. 175)
Sums of Independently Distributed Variables
• Let Sn be the sum of n independently distributed variables A1, …, An
• Each with mean and variance
• So Sn has mean  and variance2
Strong Law of Large Numbers: Limiting to Normal Distribution
• The probability density function of to normal distribution(x)
• Hence

Prob

Strong Law of Large Numbers (cont’d)
• So

Prob

(since 1- (x)  (x)/x)

Moment Generating Functions and Chernoff Bounds

Moments of Random Variable A (cont’d)
• n’th Moments of Random Variable A
• Moment generating function
Moments of Random Variable A (cont’d)
• Note

S is a formal parameter

Moments of AND of Independent Random Variables
• If A1, …, An independent with same distribution
• Interesting fact
Chernoff Bound of Random Variable A
• Uses all moments
• Uses moment generating function

By setting x = ’ (s) 1st derivative minimizes bounds

Chernoff Bound of Discrete Random Variable A
• Choose z = z0 to minimize above bound
• Need Probability Generating function
• Below mean x  

 Tails are bounded by Normal distributions

• By Chernoff Bound for p < ½
• Raghavan-Spencer bound for any  > 0

Probability Theory

Analysis of Algorithms

Prepared by

John Reif, Ph.D.