Design and analysis of computer algorithm lecture 10
1 / 58

Design and Analysis of Computer Algorithm Lecture 10 - PowerPoint PPT Presentation

  • Updated On :

Design and Analysis of Computer Algorithm Lecture 10. Pradondet Nilagupta Department of Computer Engineering. Acknowledgement.

Related searches for Design and Analysis of Computer Algorithm Lecture 10

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Design and Analysis of Computer Algorithm Lecture 10' - shawn

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Design and analysis of computer algorithm lecture 10 l.jpg

Design and Analysis of Computer AlgorithmLecture 10

Pradondet Nilagupta

Department of Computer Engineering

Acknowledgement l.jpg

  • This lecture note has been summarized from lecture note on Data Structure and Algorithm, Design and Analysis of Computer Algorithm all over the world. I can’t remember where those slide come from. However, I’d like to thank all professors who create such a good work on those lecture notes. Without those lectures, this slide can’t be finished.

The theory of np completeness l.jpg
The theory of NP-completeness

  • Tractable and intractable problems

  • NP-complete problems

Classifying problems l.jpg
Classifying problems

  • Here: Classify problems as tractable or intractable.

  • Problem is tractable if there exists a polynomial bound algorithm that solves it.

  • An algorithm is polynomial bound if its worst case growth rate can be bound by a polynomial p(n) in the size n of the problem

What constitutes reasonable time l.jpg
What constitutes reasonable time?

  • Standard working definition: polynomial time

    • On an input of size n the worst-case running time is O(nk) for some constant k

    • Polynomial time: O(n2), O(n3), O(1), O(n lg n)

    • Not in polynomial time: O(2n), O(nn), O(n!)

Polynomial time algorithms l.jpg
Polynomial-Time Algorithms

  • Are some problems solvable in polynomial time?

    • Of course: every algorithm we’ve studied provides polynomial-time solution to some problem

    • We define P to be the class of problems solvable in polynomial time

  • Are all problems solvable in polynomial time?

    • No: Turing’s “Halting Problem” is not solvable by any computer, no matter how much time is given

    • Such problems are clearly intractable, not in P

Intractable problems l.jpg
Intractable problems

  • Problem is intractable if it is not tractable.

  • Any algorithm that solves it is not polynomial bound.

  • It has a worst case growth rate f(n) which cannot be bound by a polynomial p(n) in the size n of the problem.

  • For intractable problems the bounds are:

Why is this classification useful l.jpg
Why is this classification useful?

  • If problem is intractable, no point in trying to find an efficient algorithm

  • Any algorithm too slow for large inputs.

  • To solve use approximations, heuristics, etc.

  • Sometimes we need to solve only a restricted version of the problem.

  • If restricted problem tractable design an algorithm for restricted problem

Intractable problems9 l.jpg
Intractable problems

  • Turing showed some problems so hard that no algorithm can solve them (undecidable)

  • Other researchers showed some decidable problems from automata, mathematical logic, etc. are intractable

  • These problems are so hard that they cannot be solved in polynomial time by a “nondeterministic” computer

Hard practical problems l.jpg
Hard practical problems

  • Many practical problems for which no one has yet found a polynomial bound algorithm.

  • Examples: traveling salesperson, knapsack, graph coloring, etc.

  • Most design automation problems such as testing and routing.

  • Many networks, database and graph problems.

How are they solved l.jpg
How are they solved?

  • A variety of algorithms based on backtracking, branch and bound, etc.

  • None can be shown to be polynomial bound

  • Problems can be solved by a polynomial bound verification algorithm

The theory of np completeness12 l.jpg
The theory of NP completeness

  • The theory of NP-completeness enables showing that these problems are at least as hard as NP-complete problems

  • Practical implication of knowing problem is NP-complete is that it is probably intractable ( whether it is or not has not been proved yet)

  • So any algorithm that solves it will probably be very slow for large inputs

We need to define l.jpg
We need to define

  • Decision problems

  • The class P

  • Nondeterministic algorithms

  • The class NP

  • The concept of polynomial transformations

  • The class of NP-complete problems

The theory of np completeness14 l.jpg
The theory of NP-Completeness

  • Decision problems

  • Converting optimization problems into decision problems

  • The relationship between an optimization problem and its decision version

  • The class P

  • Verification algorithms

  • The class NP

Decision problems l.jpg
Decision Problems

  • A decision problem answers yes or no for a given input

  • Examples:

  • Is there a path from s to t of length at most k?

  • Does graph G contain a Hamiltonian cycle?

A decision problem hamiltonian cycle l.jpg
A decision problem: HAMILTONIAN CYCLE

  • A hamiltonian cycle of a graph G is a cycle that includes each vertex of the graph exactly once.

  • Problem: Given a graph G, does G have a hamiltonian cycle?

Converting to decision problems l.jpg
Converting to decision problems

  • Optimization problems can be converted to decision problems (typically) by adding a bound B on the value to optimize, and asking the question:

    • Is there a solution whose value is at most B? (for a minimization problem)

    • Is there a solution whose value is at least B? (for a maximization problem)

An optimization problem traveling salesman ts l.jpg
An optimization problem: traveling salesman (TS)

  • Given:

    • A finite set C={c1,...,cm} of cities,

    • A distance function d(ci, cj) of nonnegative numbers.

  • Find the length of the minimum distance tour which includes every city exactly once

A decision problem traveling salesman l.jpg
A decision problem: traveling salesman

  • Given a finite set C={c1,...,cm} of cities, a distance function d(ci, cj) of nonnegative numbers and a bound B

  • Is there a tour of all the cities (in which each city is visited exactly once) with total length at most B?

The relation between l.jpg
The relation between

  • If we have a solution to the optimization problem we can compare the solution to the bound and answer “yes” or “no”.

  • Therefore if the optimization problem is tractable so is the decision problem

  • If the decision problem is “hard” the optimization problems are also “hard”

Class of problems p and np l.jpg
Class of Problems: P and NP

  • Definition: The class P

    • P is the class of decision problems that are polynomially bounded.

      • there exist a deterministic algorithm

  • Definition: The class NP

    • NP is the class of decision problems for which there is a polynomially bounded non-deterministic algorithm.

      • The name NP comes from “Non-deterministic Polynomially bounded.”

      • there exist a non-deterministic algorithm

  • Theorem: P NP

The goal of verification algorithms l.jpg
The goal of verification algorithms

  • The goal of a verification algorithm is to verify a “yes” answer to a decision problem’s input.

  • The inputs to the verification algorithm are the original input and a certificate (possible solution)

Example l.jpg

  • A verification algorithm for TS, verifies that a given TS tour has length at most B

A verification algorithm for path l.jpg
A verification algorithm for PATH

  • Given the problem PATH (does there exist a path of length k or less in a graph G between vertices u and v?), and a certificate p.

  • It is simple to verify that the length of p is at most k (we have to also check that p is indeed a path from u to v).

Verification algorithms l.jpg
Verification Algorithms

  • Other problems like HAMILTONIAN CYCLE are not known to have polynomial bound algorithms but given a hamiltonian cycle, it is easy to verify that the cycle is indeed hamiltonian in polynomial time.

  • A verification algorithm, takes a problem instance x and verifies it, if there exists a certificate y such that the answer for x with certificate y is “yes”

Polynomial bound verification algorithms l.jpg
Polynomial bound verification algorithms

  • Given a decision problem d.

  • A verification algorithm for d is polynomial bound if given an input x to d,

    there exists a certificate y, such that |y|=O(|x|c) where c is a constant,

    and a polynomial bound algorithm A(x, y) that verifies an answer “yes” for d with input x

The class np l.jpg
The class NP

  • NP is the class of decision problems for which there is a polynomial bounded verification algorithm

  • It can be shown that:

    • all decision problems in P, and

    • decision problems such as traveling salesman and knapsack are also in NP

A non deterministic algorithm l.jpg
A non-deterministic algorithm

  • The non-deterministic “guessing” phase.

    • Some completely arbitrary string s, “proposed solution”

    • each time the algorithm is run the string may differ

  • The deterministic “verifying” phase.

    • a deterministic algorithm takes the input of the problem and the proposed solution s, and

    • return value true or false

  • The output step.

    • If the verifying phase returned true, the algorithm outputs yes. Otherwise, there is no output.

P and np l.jpg
P and NP

  • Summary so far:

    • P = problems that can be solved in polynomial time

    • NP = problems for which a solution can be verified in polynomial time

    • Unknown whether P = NP (most suspect not)

  • Hamiltonian-cycle problem is in NP:

    • Cannot solve in polynomial time

    • Easy to verify solution in polynomial time (How?)

A problem which is in np l.jpg
A Problem Which is in NP

  • Can solve variant of TSP which is in form of a decision problem

  • TSP*: Given a complete directed graph G with cost for each edge, and an integer k. Return YES, if there is a tour with total distance  k; NO otherwise

    • Can be solved in polynomial time with nondeterministic computer

    • How?

    • Cannot be converted to polynomial time algorithm for regular computer

    • Why?

Np complete problems l.jpg
NP-Complete Problems

  • We will see that NP-Complete problems are the “hardest” problems in NP:

    • If any one NP-Complete problem can be solved in polynomial time…

    • …then every NP-Complete problem can be solved in polynomial time…

    • …and in fact every problem in NP can be solved in polynomial time (which would show P = NP)

    • Thus: solve hamiltonian-cycle in O(n100) time, you’ve proved that P = NP. Retire rich & famous.

The class np complete 1 2 l.jpg
The Class NP-Complete (1/2)

  • A problem Q is NP-complete

    • if it is in NP and

    • it is NP-hard.

  • A problem Q is NP-hard

    • if every problem in NP

    • is reducible to Q.


Let A be in NP-Complete, and B is in NP.

If A PB, then B is also NP-complete.

The class np complete 2 2 l.jpg
The Class NP-Complete (2/2)

  • A problem P is reducible to a problem Q if

    • there exists a polynomial reduction function T such that

      • For every string x,

      • if x is a yes input for P, then T(x) is a yes input for Q

      • if x is a no input for P, then T(x) is a no input for Q.

      • T can be computed in polynomially bounded time.

Class p and class relationships l.jpg
Class P and Class Relationships

  • Problems that are solvable in polynomial time on a regular computer are said to be in class P

    • All problems in P are solvable in p-time on nondeterministic computer

  • Some problems in NP are NP-complete

    • e.g., Clique problem for undirected graphs

  • All problems solvable in exponential time is an even bigger class

    • Note that all problems solvable in p-time are certainly solvable in exponential time

Theoretician s view of world l.jpg
Theoretician’s View of World

Exponential time problems

NP problems


NP-Complete problems


P problems


Polynomial reductions l.jpg
Polynomial Reductions



for Q


Yes or no



(an input

For P)

An input

for Q

  • Problem P is reducible to Q

    • P p Q

    • Transforming inputs of P to inputs of Q

  • Reducibility relation is transitive.

Reduction l.jpg

  • The crux of NP-Completeness is reducibility

    • Informally, a problem P can be reduced to another problem Q if any instance of P can be “easily rephrased” as an instance of Q, the solution to which provides a solution to the instance of P

      • What do you suppose “easily” means?

      • This rephrasing is called transformation

    • Intuitively: If P reduces to Q, P is “no harder to solve” than Q

Reducibility l.jpg

  • An example:

    • P: Given a set of Booleans, is at least one TRUE?

    • Q: Given a set of integers, is their sum positive?

    • Transformation: (x1, x2, …, xn) = (y1, y2, …, yn) where yi = 1 if xi = TRUE, yi = 0 if xi = FALSE

  • Another example:

    • Solving linear equations is reducible to solving quadratic equations

      • How can we easily use a quadratic-equation solver to solve linear equations?

Using reductions l.jpg
Using Reductions

  • If P is polynomial-time reducible to Q, we denote this P p Q

  • Definition of NP-Hard and NP-Complete:

    • If all problems R  NP are reducible to P, then P is NP-Hard

    • We say P is NP-Complete if P is NP-Hard and P  NP

  • If P p Q and P is NP-Complete, Q is alsoNP - Complete

    • This is the key idea you should take away today

Why prove np completeness l.jpg
Why Prove NP-Completeness?

  • Though nobody has proven that P != NP, if you prove a problem NP-Complete, most people accept that it is probably intractable

  • Therefore it can be important to prove that a problem is NP-Complete

    • Don’t need to come up with an efficient algorithm

    • Can instead work on approximation algorithms

Proving np completeness l.jpg
Proving NP-Completeness

  • What steps do we have to take to prove a problem Pis NP-Complete?

    • Pick a known NP-Complete problem Q

    • Reduce Q to P

      • Describe a transformation that maps instances of Q to instances of P, s.t. “yes” for P = “yes” for Q

      • Prove the transformation works

      • Prove it runs in polynomial time

    • Oh yeah, prove P NP (What if you can’t?)

Slide42 l.jpg

If you tell me that this graph is 3-colourable,

it is very difficult for me to check whether you are right.

Slide43 l.jpg

But if you tell me that this graph is 3-colorable and

give me a solution, it is very easy for me to verify whether

you are right.

Loosely speaking, problems that are difficult to compute, but

easy to verify are known as Non-deterministic Polynomial.

Cook s theorem l.jpg
Cook’s Theorem

Any NP problem can be converted to SATin polynomial time.

The sat problem l.jpg
The SAT Problem

  • One of the first problems to be proved NP-Complete was satisfiability (SAT):

    • Given a Boolean expression on n variables, can we assign values such that the expression is TRUE?

    • Ex: ((x1x2)  ((x1 x3)  x4)) x2

    • Cook’s Theorem: The satisfiability problem is NP-Complete

      • Note: Argue from first principles, not reduction

      • Proof: not here

Conjunctive normal form l.jpg
Conjunctive Normal Form

  • Even if the form of the Boolean expression is simplified, the problem may be NP-Complete

    • Literal: an occurrence of a Boolean or its negation

    • A Boolean formula is in conjunctive normal form, or CNF, if it is an AND of clauses, each of which is an OR of literals

      • Ex: (x1  x2)  (x1  x3  x4)  (x5)

    • 3-CNF: each clause has exactly 3 distinct literals

      • Ex: (x1  x2  x3)  (x1  x3  x4)  (x5  x3  x4)

      • Notice: true if at least one literal in each clause is true

The 3 cnf problem l.jpg
The 3-CNF Problem

  • Satisfiability of Boolean formulas in 3-CNF form (the 3-CNF Problem) is NP-Complete

    • Proof: Nope

  • The reason we care about the 3-CNF problem is that it is relatively easy to reduce to others

    • Thus by proving 3-CNF NP-Complete we can prove many seemingly unrelated problems NP-Complete

3 cnf clique l.jpg
3-CNF  Clique

  • What is acliqueof a graph G?

  • A: a subset of vertices fully connected to each other, i.e. a complete subgraph of G

  • The clique problem: how large is the maximum-size clique in a graph?

  • Can we turn this into a decision problem?

  • A: Yes, we call this the k-clique problem

  • Is the k-clique problem within NP?

this graph contains a 4-clique

3 cnf clique49 l.jpg
3-CNF  Clique

  • What should the reduction do?

  • A: Transform a 3-CNF formula to a graph, for which a k-clique will exist (for some k) iff the 3-CNF formula is satisfiable

3 cnf clique50 l.jpg
3-CNF  Clique

  • The reduction:

    • Let B = C1  C2  …  Ck be a 3-CNF formula with k clauses, each of which has 3 distinct literals

    • For each clause put a triple of vertices in the graph, one for each literal

    • Put an edge between two vertices if they are in different triples and their literals are consistent, meaning not each other’s negation

    • Run an example: B = (x  y  z)  (x  y  z )  (x  y  z )

3 cnf clique51 l.jpg
3-CNF  Clique

  • Prove the reduction works:

    • If B has a satisfying assignment, then each clause has at least one literal (vertex) that evaluates to 1

    • Picking one such “true” literal from each clause gives a set V’ of k vertices. V’ is a clique (Why?)

    • If G has a clique V’ of size k, it must contain one vertex in each triple (clause) (Why?)

    • We can assign 1 to each literal corresponding with a vertex in V’, without fear of contradiction

Clique vertex cover l.jpg
Clique  Vertex Cover

  • A vertex cover for a graph G is a set of vertices incident to every edge in G

  • The vertex cover problem: what is the minimum size vertex cover in G?

  • Restated as a decision problem: does a vertex cover of size k exist in G?

  • Thm : vertex cover is NP-Complete

Clique vertex cover53 l.jpg
Clique  Vertex Cover

  • First, show vertex cover in NP(How?)

  • Next, reduce k-clique to vertex cover

    • The complement GC of a graph G contains exactly those edges not in G

    • Compute GC in polynomial time

    • G has a clique of size k iff GC has a vertex cover of size |V| - k

Clique vertex cover54 l.jpg
Clique  Vertex Cover

  • Claim: If G has a clique of size k,GC has a vertex cover of size |V| - k

    • Let V’ be the k-clique

    • Then V - V’ is a vertex cover in GC

      • Let (u,v) be any edge in GC

      • Then u and v cannot both be in V’ (Why?)

      • Thus at least one of u or v is in V-V’ (why?), so edge (u, v) is covered by V-V’

      • Since true for any edge in GC, V-V’ is a vertex cover

Clique vertex cover55 l.jpg
Clique  Vertex Cover

  • Claim: If GC has a vertex cover V’  V, with |V’| = |V| - k, then G has a clique of size k

    • For all u,v V, if (u,v)  GC then u  V’ or v  V’ or both (Why?)

    • Contrapositive: if u  V’ and v  V’, then (u,v)  E

    • In other words, all vertices in V-V’ are connected by an edge, thus V-V’ is a clique

    • Since |V| - |V’| = k, the size of the clique is k

General comments l.jpg
General Comments

  • Literally hundreds of problems have been shown to be NP-Complete

  • Some reductions are profound, some are comparatively easy, many are easy once the key insight is given

  • You can expect a simple NP-Completeness proof on the final

Example np complete problems 1 2 l.jpg
Example:NP-Complete Problems(1/2)

Vertex Cover (VC)

Instance: Graph G=(V,E) and integer k

Question: Does there exist a vertex cover of size at most k?

(V’ V is a vertex cover if for each (u,v)E, either uV’or vV’).

Independent Set (IND)

Instance: Graph G=(V,E) and integer k

Question: Does G have an independent set of size at least k?

(V’ V is an independent set if for any u,vV’, (u,v)E.)


Instance: Graph G=(V,E) and integer k

Question: Does G have a clique of size at least k?

(V’ V is a clique if for any u,vV’, (u,v)E.)

Example np complete problems 2 2 l.jpg
Example:NP-Complete Problems(2/2)

Hamiltonian Path (HP)

Instance: Graph G=(V,E) and integer k

Question: Does there exist a Hamiltonian Path of G?

That is, does  a simple path of length |V| - 1?

3 Color Problem (3COL)

Instance: Graph G=(V,E)

Question: Can we color the nodes of G with three three colors such that no two adjacent nodes of G have the same color?


Instance: Given a set of integers

Question: does there exist a subset that adds up to some target T?


Bin Packing

Integer Linear Programming (ILP)