Randomized algorithms
1 / 48

Randomized Algorithms - PowerPoint PPT Presentation

  • Uploaded on

Randomized Algorithms. Morteza ZadiMoghaddam Amin Sayedi. Types of Randomized algorithms. Las Vegas Monte Carlo. Las Vegas. Always gives the true answer. Running time is random. Running time is bounded. Quick sort is a Las Vegas algorithm. Monte Carlo. It may produce incorrect answer!

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Randomized Algorithms' - ailis

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Randomized algorithms

Randomized Algorithms

Morteza ZadiMoghaddam

Amin Sayedi

Types of randomized algorithms
Types of Randomized algorithms

  • Las Vegas

  • Monte Carlo

Las vegas
Las Vegas

  • Always gives the true answer.

  • Running time is random.

  • Running time is bounded.

  • Quick sort is a Las Vegas algorithm.

Monte carlo
Monte Carlo

  • It may produce incorrect answer!

  • We are able to bound its probability.

  • By running it many times on independent random variables, we can make the failure probability arbitrarily small at the expense of running time.

Monte carlo example
Monte Carlo Example

  • Suppose we want to find a number among n given numbers which is larger than or equal to the median.

Monte carlo example1
Monte Carlo Example

Suppose A1 < … < An.

We want Ai, such that i ≥ n/2.

It’s obvious that the best deterministic algorithm needs O(n) time to produce the answer.

n may be very large!

Suppose n is 100,000,000,000 !

Monte carlo example2
Monte Carlo Example

  • Choose 100 of the numbers with equal probability.

  • find the maximum among these numbers.

  • Return the maximum.

Monte carlo example3
Monte Carlo Example

  • The running time of the given algorithm is O(1).

  • The probability of Failure is 1/(2100).

  • Consider that the algorithm may return a wrong answer but the probability is very smaller than the hardware failure or even an earthquake!

Monte carlo1
Monte Carlo

  • Suppose the output is Yes or No.

  • One sided error.

  • Two sided error.

Rp class randomized polynomial
RP Class( randomized polynomial )

  • Bounded polynomial time in the worst case.

  • If the answer is Yes; Pr[ return Yes] > ½.

  • If the answer is No; Pr[ return Yes] = 0.

  • ½ is not actually important.

Pp class probabilistic polynomial
PP Class( probabilistic polynomial )

  • Bounded polynomial time in worst case.

  • If the answer is Yes; Pr[ return Yes] > ½.

  • If the answer is No; Pr[ return Yes] < ½.

  • Unfortunately the definition is weak because the distance to ½ is important but is not considered.

Routing problem
Routing Problem

  • There are n computers.

  • Each computer has a packet.

  • Each packet has a destination D(i).

  • Packets can not follow the same edge simultaneously.

  • An oblivious algorithm is required.

Routing problem1
Routing Problem

  • For any deterministic oblivious algorithm on a network of N nodes each of outdegree d, there is an instance of permutation routing requiring (N/d) ½.

Routing problem2
Routing Problem

  • Pick random intermediate destination.

  • Packet i first travels to the intermediate destination and then to the final destination.

  • With probability at least 1-(1/N), every packet reaches its destination in 14n of fewer steps in Qn.

  • The expected number of steps is 15n.

Maximum satisfiability
Maximum Satisfiability

  • You have m clauses and n boolean variables.

  • Each clause contains some of variables or some of complements.

  • A clause is satisfied if at least one of it’s variables are satisfied.

  • We want to set the variables such that the number of satisfied clauses is maximized.

Example for maximum sat
Example for Maximum Sat

  • There are 3 variables A, B and C.

  • M1 = (A) or (B)

  • M2 = (A) or (not B) or (not C)

  • M3 = (C)

  • M4 = (B) or (not C)

  • M5 = (not C)

Example of maximum sat
Example of Maximum Sat

  • Set A = True

  • Set B = True

  • Set C = False

  • Four of the clauses are satisfied.

Maximum sat
Maximum Sat

  • This problem is a famous problem which has no polynomial time algorithm yet. It’s NP-hard.

Maximum sat1
Maximum Sat

  • For any set of m clauses, there is truth assignment for the variables that satisfies at least m/2 clauses.

Maximum sat2
Maximum Sat

  • Let Zi =1 if the i-th clause is satisfied and 0 otherwise.

  • Set the variables in a random way.

  • The probability of a clause with k variables to be true is 1- (1/(2k)) >= ½.

  • So E[Z1]+…+E[Zm] >= ½.

  • Thus there exist at least one assignment such that Z1+…+Zm >= ½.

Maximum sat algorithm
Maximum Sat algorithm

  • This problem is NP-hard so we seek for approximation algorithms.

  • We have an algorithm that produces an answer which is at least ½ of the best answer.

  • If all clauses have at least 2 literals then we have an algorithm that produces an answer which is at least ¾ of the best one.

Maximum sat algorithm1
Maximum Sat algorithm

  • We want to maximize Z1+…+Zm.

  • We have some inequalities:

  • ∑ yi (if Xi is in Zj and is uncomplemented)

  • ∑ (1-yi) (if Xi is in Zj and is complemented).

  • This inequality must be hold:

  • ∑ yi + ∑ (1-yi) >= Zj

  • This problem could be solved using integer linear programming.

  • We have to use linear programming.

Maximum sat3
Maximum Sat

  • Solve the problem using linear programming.

  • You get a real number for each yi or zi.

  • Assign Xi true with the probability yi.

  • The expected number of clauses that are satisfied is (1- 1/e) of the best answer.

Maximum sat algorithm2
Maximum Sat algorithm

  • Using both algorithms and choosing the better answer gives us an answer which is at least ¾ of the best answer!!! Which is better than ½ and 1- 1/e.

2 sat

  • Every clause has at most 2 literals.

  • We want to check if all clauses can be satisfied.

  • It has polynomial algorithm.

  • Assign random values to the variables.

  • If all clauses are satisfied we are finished.

  • If there is an unsatisfied clause, the value of one of it’s literals is different from the best answer.

  • Change the value of one of the variables in this clause. You may make a good change or bad one.

2 sat1

  • You are walking on a path.

  • If you are on 0 you go to 1.

  • If you are on i you go to i+1 or i-1 with equal probability.

  • The expected number of steps to reach the end of the path is O(n2).

  • So the given algorithm is O(n3).

Graph connectivity
Graph Connectivity

  • You want to check if two vertices u and v are in the same connected component.

  • Start a random walk from v.

  • Have a random walk of length 2n3.

  • If you haven’t visited u, the probability of u to be in this component is less than ½.

  • By repeating this algorithm, you can make the probability of failure arbitrarily small.

Graph connectivity1
Graph Connectivity

  • Running time of algorithm is O(n3).

  • Required space is O(logn).

Diameter of a point set
Diameter of a Point Set

  • You want to find the diameter of set of n points in the space.

  • Suppose I(x) is the convex body formed by the intersection of n sphere centered at n points with radius x.

  • F(p) is distance between p and the point in the set that is farthest from p.

Diameter of a point set1
Diameter of a Point Set

  • Consider I(x) when x=F(p).

  • For any q in S, if q is in I(x) then F(q)<F(p).

  • And if q is not in I(x) then F(p)<F(q).

Diameter of a point set2
Diameter of a Point Set

  • Pick a point p in s at random. Computer F(p). [O(n)]

  • Set x=F(p). Compute I(x). [O(n logn)]

  • Find the points outside I(x). Call this set T. [O(n logn)]

  • If T is empty return x as the answer, else continue on T.

Diameter of a point set3
Diameter of a Point Set

  • The running time of algorithm above is O(n log n).

  • In each step, all points that have smaller F(x) than the chosen point are removed.

All pairs shortest paths
All-Pairs Shortest Paths

  • Let G(V,E) be an undirected, connected graph with V={1,…,n} and |E|=m.

  • The adjacency matrix A is an n  n 0-1 matrix with Aij=Aji=1 if and only if the edge (i,j) is present in E.

  • We are going to compute matrix D which Dij equals the length of a shortest path from vertex i to vertex j.

All pairs distances
All-Pairs Distances

  • Z A2

  • Compute matrix A’ such that A’ij=1 if and only if i ≠ j and (Aij=1 or Zij>0)

  • If A’ij=1 for all i ≠ j then return D = 2A’-A.

  • Recursively compute the APD matrix D’ for the graph G’ with adjacency matrix A’.

  • S AD’

  • Return matrix D with Dij=2D’ij if Sij≥D’ijZii , otherwise Dij=2D’ij-1.

Randomized algorithms

  • The APD algorithm computes the distance matrix for an n-vertex graph in time O(MM(n)log(n)) using integer matrix multiplication algorithm.

  • Matrix multiplication algorithm running in time O(n2.376).

Boolean product witness matrix
Boolean Product Witness Matrix

  • Suppose A and B are nn boolean matrices and P=AB is their product under Boolean matrix multiplication.

  • A witness for Pij is an index k  {1,…,n} such that Aik=Akj=1. Observe that Pij=1 if and only if it has some witness k.

Randomized algorithms

  • W -AB

  • 2. for t=0,…, log(n) do

  • 2.1. r 2t

  • 2.2. Repeat 3.77log(n) times

    • 2.2.1. Choose random R  {1,…,n} with |R|=r .

    • 2.2.2. Compute AR and BR .

    • 2.2.3. Z ARBR .

    • 2.2.4. for all (i,j) do

      • if Wij < 0 and Zij is witness then Wij Zij

  • 3. for all (i,j) do

    • if Wij < 0 then find witness Wij by brute force.

Randomized algorithms

  • The BPWM algorithm is a Las Vegas algorithm for the BPWM problem with expected running time O(MM(n)log2(n)).

  • The probability that no witness is found for Pij before the end of Step 2 is at most

  • (1-1/2e)3.77log(n) 1/n.

Determining shortest path
Determining Shortest Path

  • A successor matrix S for an n-vertex graph G is an n  n matrix such that Sij is the index of a neighbor of vertex i that lies on a shortest path from i to j.

Randomized algorithms

  • Compute the distance matrix D=APD(A).

  • for s={0,1,2} do

    • Compute 0-1 matrix Dkj(s)=1 if and only if Dkj+1 = s (mod 3)

    • Compute the witness matrix W(s)=BPWM(A,D(s)).

  • Compute successor matrix S for G.

Randomized algorithms

  • Algorithm APSP computes the successor matrix for an n-vertex graph G in expected time O(MM(n)log2(n)).

Algorithm contract
Algorithm contract

  • H G

  • While H has more than 2 vertices do

    • Choose an edge (x,y) uniformly at random from the edges in H.

    • F F  {(x,y)}.

    • H H / (x,y).

  • (C,V/C) the sets of vertices corresponding to the two meta-vertices in H=G/F.


  • n |v|

  • if n  6 then compute min-cut of G by brute-force enumeration else

    • t 1+n/2

    • Using Algorithm Contract, perform two independent contraction sequences to obtain graphs H1 and H2 each with t vertices.

    • Recursively compute min-cuts in each of H1 and H2.

    • Return the smaller of the two min-cuts.


  • Algorithm Fastcut succeeds in finding a min-cut with probability (1/log(n)).

  • Algorithm Fastcut runs in O(n2log(n)) time and uses O(n2) space.

Randomized algorithms

  • Finding MST in a graph with n vertices and m edges has a Las Vegas algorithm which has the expected running time O(n+m).

  • But We don’t have enough time to Explain it !!!

Research problems
Research problems

  • Devise an algorithm for the all-pairs shortest paths problem that does not use matrix multiplication and runs in time O(n3-) for a positive constant .

  • Devise an algorithm for computing the diameter of an unweighted graph that does not use matrix multiplication and runs in time O(n3-) for a positive constant .

Research problems1
Research problems

  • Devise a Las Vegas or a deterministic algorithm for min-cuts with running time close to O(n2).

  • Is there a randomized algorithm for min-cuts with expected running time close to O(m)?