probability theory overview and analysis of randomized algorithms l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Probability Theory Overview and Analysis of Randomized Algorithms PowerPoint Presentation
Download Presentation
Probability Theory Overview and Analysis of Randomized Algorithms

Loading in 2 Seconds...

play fullscreen
1 / 50

Probability Theory Overview and Analysis of Randomized Algorithms - PowerPoint PPT Presentation


  • 199 Views
  • Uploaded on

Probability Theory Overview and Analysis of Randomized Algorithms. Analysis of Algorithms. Prepared by John Reif, Ph.D. Probability Theory Topics . Random Variables: Binomial and Geometric Useful Probabilistic Bounds and Inequalities. Readings. Main Reading Selections:

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Probability Theory Overview and Analysis of Randomized Algorithms' - naava


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
probability theory overview and analysis of randomized algorithms

Probability Theory Overview and Analysis of Randomized Algorithms

Analysis of Algorithms

Prepared by

John Reif, Ph.D.

probability theory topics
Probability Theory Topics
  • Random Variables: Binomial and Geometric
  • Useful Probabilistic Bounds and Inequalities
readings
Readings
  • Main Reading Selections:
    • CLR, Chapter 5 and Appendix C
probability measures
Probability Measures
  • A probability measure (Prob) is a mapping from a set of events to the reals such that:
    • For any event A

0  Prob(A)  1

    • Prob (all possible events) = 1
    • If A, B are mutually exclusive events, then

Prob(A  B) = Prob (A) + Prob (B)

bayes theorem
Bayes’ Theorem
  • If A1, …, An are mutually exclusive and contain all events then
random variable a cont d
Random Variable A (cont’d)
  • Prob Distribution Function
random variable a cont d9
Random Variable A (cont’d)
  • If for Random Variables A,B
  • Then “A upper bounds B” and “B lower bounds A”
expectation of random variable a
Expectation of Random Variable A
  • Ā is also called “average of A” and “mean of A” = A
pair wise independent random variables
Pair-Wise Independent Random Variables
  • A,B independent if

Prob(A  B) = Prob(A) X Prob(B)

  • Equivalent definition of independence
bounding numbers of permutations
Bounding Numbers of Permutations
  • n! = n x (n-1) x 2 x 1= number of permutations of n objects
  • Stirling’s formula

n! = f(n) x (1+o(1)), where

bounding numbers of permutations cont d
Bounding Numbers of Permutations (cont’d)
  • Note
    • Tighter bound

= number of permutations of n objects taken k at a time

bounding numbers of combinations
Bounding Numbers of Combinations

= number of (unordered) combinations of n objects taken k at a time

  • Bounds (due to Erdos & Spencer, p. 18)
bernoulli variable
Bernoulli Variable
  • Ai is 1 with prob P and 0 with prob 1-P
  • Binomial Variable
    • B is sum of n independent Bernoulli variables Ai each with some probability p
poisson trial
Poisson Trial
  • Ai is 1 with prob Piand 0 with prob 1-Pi
  • Suppose B’ is the sum of n independent Poisson trialsAi with probability Pi for i > 1, …, n
hoeffding s theorem
Hoeffding’s Theorem
  • B’ is upper bounded by a Binomial Variable B
  • Parameters n,p where
geometric variable v

procedure

GEOMETRIC parameter p

¬

begin

V 0

loop with probability 1-p

goto

exit

Geometric Variable V
  • parameter p
probabilistic inequalities
Probabilistic Inequalities
  • For Random Variable A
markov and chebychev probabilistic inequalities
Markov and Chebychev Probabilistic Inequalities
  • Markov Inequality (uses only mean)
  • Chebychev Inequality (uses mean and variance)
example of use of markov and chebychev probabilistic inequalities
Example of use of Markov and Chebychev Probabilistic Inequalities
  • If B is a Binomial with parameters n,p
normal distribution
Normal Distribution
  • Bounds x > 0 (Feller, p. 175)
sums of independently distributed variables
Sums of Independently Distributed Variables
  • Let Sn be the sum of n independently distributed variables A1, …, An
  • Each with mean and variance
  • So Sn has mean  and variance2
strong law of large numbers limiting to normal distribution
Strong Law of Large Numbers: Limiting to Normal Distribution
  • The probability density function of to normal distribution(x)
  • Hence

Prob

strong law of large numbers cont d
Strong Law of Large Numbers (cont’d)
  • So

Prob

(since 1- (x)  (x)/x)

advanced material
Advanced Material

Moment Generating Functions and Chernoff Bounds

moments of random variable a cont d
Moments of Random Variable A (cont’d)
  • n’th Moments of Random Variable A
  • Moment generating function
moments of random variable a cont d37
Moments of Random Variable A (cont’d)
  • Note

S is a formal parameter

moments of and of independent random variables
Moments of AND of Independent Random Variables
  • If A1, …, An independent with same distribution
chernoff bound of random variable a
Chernoff Bound of Random Variable A
  • Uses all moments
  • Uses moment generating function

By setting x = ’ (s) 1st derivative minimizes bounds

chernoff bound of discrete random variable a
Chernoff Bound of Discrete Random Variable A
  • Choose z = z0 to minimize above bound
  • Need Probability Generating function
anguin valiant s bounds for binomial b with parameters n p cont d
Anguin-Valiant’s Bounds for Binomial B with Parameters n,p (cont’d)

 Tails are bounded by Normal distributions

binomial variable b with parameters p n and expectation pn
Binomial Variable B with Parameters p,N and Expectation  = pN
  • By Chernoff Bound for p < ½
  • Raghavan-Spencer bound for any  > 0
probability theory

Probability Theory

Analysis of Algorithms

Prepared by

John Reif, Ph.D.