Loading in 5 sec....

Design and Analysis of Computer Algorithm Lecture 10PowerPoint Presentation

Design and Analysis of Computer Algorithm Lecture 10

- By
**shawn** - Follow User

- 214 Views
- Updated On :

Design and Analysis of Computer Algorithm Lecture 10. Pradondet Nilagupta Department of Computer Engineering. Acknowledgement.

Related searches for Design and Analysis of Computer Algorithm Lecture 10

Download Presentation
## PowerPoint Slideshow about 'Design and Analysis of Computer Algorithm Lecture 10' - shawn

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

### Design and Analysis of Computer AlgorithmLecture 10

Pradondet Nilagupta

Department of Computer Engineering

Acknowledgement

- This lecture note has been summarized from lecture note on Data Structure and Algorithm, Design and Analysis of Computer Algorithm all over the world. I can’t remember where those slide come from. However, I’d like to thank all professors who create such a good work on those lecture notes. Without those lectures, this slide can’t be finished.

The theory of NP-completeness

- Tractable and intractable problems
- NP-complete problems

Classifying problems

- Here: Classify problems as tractable or intractable.
- Problem is tractable if there exists a polynomial bound algorithm that solves it.
- An algorithm is polynomial bound if its worst case growth rate can be bound by a polynomial p(n) in the size n of the problem

What constitutes reasonable time?

- Standard working definition: polynomial time
- On an input of size n the worst-case running time is O(nk) for some constant k
- Polynomial time: O(n2), O(n3), O(1), O(n lg n)
- Not in polynomial time: O(2n), O(nn), O(n!)

Polynomial-Time Algorithms

- Are some problems solvable in polynomial time?
- Of course: every algorithm we’ve studied provides polynomial-time solution to some problem
- We define P to be the class of problems solvable in polynomial time

- Are all problems solvable in polynomial time?
- No: Turing’s “Halting Problem” is not solvable by any computer, no matter how much time is given
- Such problems are clearly intractable, not in P

Intractable problems

- Problem is intractable if it is not tractable.
- Any algorithm that solves it is not polynomial bound.
- It has a worst case growth rate f(n) which cannot be bound by a polynomial p(n) in the size n of the problem.
- For intractable problems the bounds are:

Why is this classification useful?

- If problem is intractable, no point in trying to find an efficient algorithm
- Any algorithm too slow for large inputs.
- To solve use approximations, heuristics, etc.
- Sometimes we need to solve only a restricted version of the problem.
- If restricted problem tractable design an algorithm for restricted problem

Intractable problems

- Turing showed some problems so hard that no algorithm can solve them (undecidable)
- Other researchers showed some decidable problems from automata, mathematical logic, etc. are intractable
- These problems are so hard that they cannot be solved in polynomial time by a “nondeterministic” computer

Hard practical problems

- Many practical problems for which no one has yet found a polynomial bound algorithm.
- Examples: traveling salesperson, knapsack, graph coloring, etc.
- Most design automation problems such as testing and routing.
- Many networks, database and graph problems.

How are they solved?

- A variety of algorithms based on backtracking, branch and bound, etc.
- None can be shown to be polynomial bound
- Problems can be solved by a polynomial bound verification algorithm

The theory of NP completeness

- The theory of NP-completeness enables showing that these problems are at least as hard as NP-complete problems
- Practical implication of knowing problem is NP-complete is that it is probably intractable ( whether it is or not has not been proved yet)
- So any algorithm that solves it will probably be very slow for large inputs

We need to define

- Decision problems
- The class P
- Nondeterministic algorithms
- The class NP
- The concept of polynomial transformations
- The class of NP-complete problems

The theory of NP-Completeness

- Decision problems
- Converting optimization problems into decision problems
- The relationship between an optimization problem and its decision version
- The class P
- Verification algorithms
- The class NP

Decision Problems

- A decision problem answers yes or no for a given input
- Examples:
- Is there a path from s to t of length at most k?
- Does graph G contain a Hamiltonian cycle?

A decision problem: HAMILTONIAN CYCLE

- A hamiltonian cycle of a graph G is a cycle that includes each vertex of the graph exactly once.
- Problem: Given a graph G, does G have a hamiltonian cycle?

Converting to decision problems

- Optimization problems can be converted to decision problems (typically) by adding a bound B on the value to optimize, and asking the question:
- Is there a solution whose value is at most B? (for a minimization problem)
- Is there a solution whose value is at least B? (for a maximization problem)

An optimization problem: traveling salesman (TS)

- Given:
- A finite set C={c1,...,cm} of cities,
- A distance function d(ci, cj) of nonnegative numbers.

- Find the length of the minimum distance tour which includes every city exactly once

A decision problem: traveling salesman

- Given a finite set C={c1,...,cm} of cities, a distance function d(ci, cj) of nonnegative numbers and a bound B
- Is there a tour of all the cities (in which each city is visited exactly once) with total length at most B?

The relation between

- If we have a solution to the optimization problem we can compare the solution to the bound and answer “yes” or “no”.
- Therefore if the optimization problem is tractable so is the decision problem
- If the decision problem is “hard” the optimization problems are also “hard”

Class of Problems: P and NP

- Definition: The class P
- P is the class of decision problems that are polynomially bounded.
- there exist a deterministic algorithm

- P is the class of decision problems that are polynomially bounded.
- Definition: The class NP
- NP is the class of decision problems for which there is a polynomially bounded non-deterministic algorithm.
- The name NP comes from “Non-deterministic Polynomially bounded.”
- there exist a non-deterministic algorithm

- NP is the class of decision problems for which there is a polynomially bounded non-deterministic algorithm.
- Theorem: P NP

The goal of verification algorithms

- The goal of a verification algorithm is to verify a “yes” answer to a decision problem’s input.
- The inputs to the verification algorithm are the original input and a certificate (possible solution)

Example

- A verification algorithm for TS, verifies that a given TS tour has length at most B

A verification algorithm for PATH

- Given the problem PATH (does there exist a path of length k or less in a graph G between vertices u and v?), and a certificate p.
- It is simple to verify that the length of p is at most k (we have to also check that p is indeed a path from u to v).

Verification Algorithms

- Other problems like HAMILTONIAN CYCLE are not known to have polynomial bound algorithms but given a hamiltonian cycle, it is easy to verify that the cycle is indeed hamiltonian in polynomial time.
- A verification algorithm, takes a problem instance x and verifies it, if there exists a certificate y such that the answer for x with certificate y is “yes”

Polynomial bound verification algorithms

- Given a decision problem d.
- A verification algorithm for d is polynomial bound if given an input x to d,
there exists a certificate y, such that |y|=O(|x|c) where c is a constant,

and a polynomial bound algorithm A(x, y) that verifies an answer “yes” for d with input x

The class NP

- NP is the class of decision problems for which there is a polynomial bounded verification algorithm
- It can be shown that:
- all decision problems in P, and
- decision problems such as traveling salesman and knapsack are also in NP

A non-deterministic algorithm

- The non-deterministic “guessing” phase.
- Some completely arbitrary string s, “proposed solution”
- each time the algorithm is run the string may differ

- The deterministic “verifying” phase.
- a deterministic algorithm takes the input of the problem and the proposed solution s, and
- return value true or false

- The output step.
- If the verifying phase returned true, the algorithm outputs yes. Otherwise, there is no output.

P and NP

- Summary so far:
- P = problems that can be solved in polynomial time
- NP = problems for which a solution can be verified in polynomial time
- Unknown whether P = NP (most suspect not)

- Hamiltonian-cycle problem is in NP:
- Cannot solve in polynomial time
- Easy to verify solution in polynomial time (How?)

A Problem Which is in NP

- Can solve variant of TSP which is in form of a decision problem
- TSP*: Given a complete directed graph G with cost for each edge, and an integer k. Return YES, if there is a tour with total distance k; NO otherwise
- Can be solved in polynomial time with nondeterministic computer
- How?
- Cannot be converted to polynomial time algorithm for regular computer
- Why?

NP-Complete Problems

- We will see that NP-Complete problems are the “hardest” problems in NP:
- If any one NP-Complete problem can be solved in polynomial time…
- …then every NP-Complete problem can be solved in polynomial time…
- …and in fact every problem in NP can be solved in polynomial time (which would show P = NP)
- Thus: solve hamiltonian-cycle in O(n100) time, you’ve proved that P = NP. Retire rich & famous.

The Class NP-Complete (1/2)

- A problem Q is NP-complete
- if it is in NP and
- it is NP-hard.

- A problem Q is NP-hard
- if every problem in NP
- is reducible to Q.

Theorem:

Let A be in NP-Complete, and B is in NP.

If A PB, then B is also NP-complete.

The Class NP-Complete (2/2)

- A problem P is reducible to a problem Q if
- there exists a polynomial reduction function T such that
- For every string x,
- if x is a yes input for P, then T(x) is a yes input for Q
- if x is a no input for P, then T(x) is a no input for Q.
- T can be computed in polynomially bounded time.

- there exists a polynomial reduction function T such that

Class P and Class Relationships

- Problems that are solvable in polynomial time on a regular computer are said to be in class P
- All problems in P are solvable in p-time on nondeterministic computer

- Some problems in NP are NP-complete
- e.g., Clique problem for undirected graphs

- All problems solvable in exponential time is an even bigger class
- Note that all problems solvable in p-time are certainly solvable in exponential time

Theoretician’s View of World

Exponential time problems

NP problems

TOH

NP-Complete problems

TSP*

P problems

SORTING

Polynomial Reductions

T(x)

Algorithm

for Q

T

Yes or no

Answer

x

(an input

For P)

An input

for Q

- Problem P is reducible to Q
- P p Q
- Transforming inputs of P to inputs of Q

- Reducibility relation is transitive.

Reduction

- The crux of NP-Completeness is reducibility
- Informally, a problem P can be reduced to another problem Q if any instance of P can be “easily rephrased” as an instance of Q, the solution to which provides a solution to the instance of P
- What do you suppose “easily” means?
- This rephrasing is called transformation

- Intuitively: If P reduces to Q, P is “no harder to solve” than Q

- Informally, a problem P can be reduced to another problem Q if any instance of P can be “easily rephrased” as an instance of Q, the solution to which provides a solution to the instance of P

Reducibility

- An example:
- P: Given a set of Booleans, is at least one TRUE?
- Q: Given a set of integers, is their sum positive?
- Transformation: (x1, x2, …, xn) = (y1, y2, …, yn) where yi = 1 if xi = TRUE, yi = 0 if xi = FALSE

- Another example:
- Solving linear equations is reducible to solving quadratic equations
- How can we easily use a quadratic-equation solver to solve linear equations?

- Solving linear equations is reducible to solving quadratic equations

Using Reductions

- If P is polynomial-time reducible to Q, we denote this P p Q
- Definition of NP-Hard and NP-Complete:
- If all problems R NP are reducible to P, then P is NP-Hard
- We say P is NP-Complete if P is NP-Hard and P NP

- If P p Q and P is NP-Complete, Q is alsoNP - Complete
- This is the key idea you should take away today

Why Prove NP-Completeness?

- Though nobody has proven that P != NP, if you prove a problem NP-Complete, most people accept that it is probably intractable
- Therefore it can be important to prove that a problem is NP-Complete
- Don’t need to come up with an efficient algorithm
- Can instead work on approximation algorithms

Proving NP-Completeness

- What steps do we have to take to prove a problem Pis NP-Complete?
- Pick a known NP-Complete problem Q
- Reduce Q to P
- Describe a transformation that maps instances of Q to instances of P, s.t. “yes” for P = “yes” for Q
- Prove the transformation works
- Prove it runs in polynomial time

- Oh yeah, prove P NP (What if you can’t?)

If you tell me that this graph is 3-colourable,

it is very difficult for me to check whether you are right.

But if you tell me that this graph is 3-colorable and

give me a solution, it is very easy for me to verify whether

you are right.

Loosely speaking, problems that are difficult to compute, but

easy to verify are known as Non-deterministic Polynomial.

Cook’s Theorem

Any NP problem can be converted to SATin polynomial time.

The SAT Problem

- One of the first problems to be proved NP-Complete was satisfiability (SAT):
- Given a Boolean expression on n variables, can we assign values such that the expression is TRUE?
- Ex: ((x1x2) ((x1 x3) x4)) x2
- Cook’s Theorem: The satisfiability problem is NP-Complete
- Note: Argue from first principles, not reduction
- Proof: not here

Conjunctive Normal Form

- Even if the form of the Boolean expression is simplified, the problem may be NP-Complete
- Literal: an occurrence of a Boolean or its negation
- A Boolean formula is in conjunctive normal form, or CNF, if it is an AND of clauses, each of which is an OR of literals
- Ex: (x1 x2) (x1 x3 x4) (x5)

- 3-CNF: each clause has exactly 3 distinct literals
- Ex: (x1 x2 x3) (x1 x3 x4) (x5 x3 x4)
- Notice: true if at least one literal in each clause is true

The 3-CNF Problem

- Satisfiability of Boolean formulas in 3-CNF form (the 3-CNF Problem) is NP-Complete
- Proof: Nope

- The reason we care about the 3-CNF problem is that it is relatively easy to reduce to others
- Thus by proving 3-CNF NP-Complete we can prove many seemingly unrelated problems NP-Complete

3-CNF Clique

- What is acliqueof a graph G?
- A: a subset of vertices fully connected to each other, i.e. a complete subgraph of G
- The clique problem: how large is the maximum-size clique in a graph?
- Can we turn this into a decision problem?
- A: Yes, we call this the k-clique problem
- Is the k-clique problem within NP?

this graph contains a 4-clique

3-CNF Clique

- What should the reduction do?
- A: Transform a 3-CNF formula to a graph, for which a k-clique will exist (for some k) iff the 3-CNF formula is satisfiable

3-CNF Clique

- The reduction:
- Let B = C1 C2 … Ck be a 3-CNF formula with k clauses, each of which has 3 distinct literals
- For each clause put a triple of vertices in the graph, one for each literal
- Put an edge between two vertices if they are in different triples and their literals are consistent, meaning not each other’s negation
- Run an example: B = (x y z) (x y z ) (x y z )

3-CNF Clique

- Prove the reduction works:
- If B has a satisfying assignment, then each clause has at least one literal (vertex) that evaluates to 1
- Picking one such “true” literal from each clause gives a set V’ of k vertices. V’ is a clique (Why?)
- If G has a clique V’ of size k, it must contain one vertex in each triple (clause) (Why?)
- We can assign 1 to each literal corresponding with a vertex in V’, without fear of contradiction

Clique Vertex Cover

- A vertex cover for a graph G is a set of vertices incident to every edge in G
- The vertex cover problem: what is the minimum size vertex cover in G?
- Restated as a decision problem: does a vertex cover of size k exist in G?
- Thm : vertex cover is NP-Complete

Clique Vertex Cover

- First, show vertex cover in NP(How?)
- Next, reduce k-clique to vertex cover
- The complement GC of a graph G contains exactly those edges not in G
- Compute GC in polynomial time
- G has a clique of size k iff GC has a vertex cover of size |V| - k

Clique Vertex Cover

- Claim: If G has a clique of size k,GC has a vertex cover of size |V| - k
- Let V’ be the k-clique
- Then V - V’ is a vertex cover in GC
- Let (u,v) be any edge in GC
- Then u and v cannot both be in V’ (Why?)
- Thus at least one of u or v is in V-V’ (why?), so edge (u, v) is covered by V-V’
- Since true for any edge in GC, V-V’ is a vertex cover

Clique Vertex Cover

- Claim: If GC has a vertex cover V’ V, with |V’| = |V| - k, then G has a clique of size k
- For all u,v V, if (u,v) GC then u V’ or v V’ or both (Why?)
- Contrapositive: if u V’ and v V’, then (u,v) E
- In other words, all vertices in V-V’ are connected by an edge, thus V-V’ is a clique
- Since |V| - |V’| = k, the size of the clique is k

General Comments

- Literally hundreds of problems have been shown to be NP-Complete
- Some reductions are profound, some are comparatively easy, many are easy once the key insight is given
- You can expect a simple NP-Completeness proof on the final

Example:NP-Complete Problems(1/2)

Vertex Cover (VC)

Instance: Graph G=(V,E) and integer k

Question: Does there exist a vertex cover of size at most k?

(V’ V is a vertex cover if for each (u,v)E, either uV’or vV’).

Independent Set (IND)

Instance: Graph G=(V,E) and integer k

Question: Does G have an independent set of size at least k?

(V’ V is an independent set if for any u,vV’, (u,v)E.)

Clique

Instance: Graph G=(V,E) and integer k

Question: Does G have a clique of size at least k?

(V’ V is a clique if for any u,vV’, (u,v)E.)

Example:NP-Complete Problems(2/2)

Hamiltonian Path (HP)

Instance: Graph G=(V,E) and integer k

Question: Does there exist a Hamiltonian Path of G?

That is, does a simple path of length |V| - 1?

3 Color Problem (3COL)

Instance: Graph G=(V,E)

Question: Can we color the nodes of G with three three colors such that no two adjacent nodes of G have the same color?

Subset-sum

Instance: Given a set of integers

Question: does there exist a subset that adds up to some target T?

Partition

Bin Packing

Integer Linear Programming (ILP)

Download Presentation

Connecting to Server..