INHERENT LIMITATIONS OF COMPUTER PROGRAMS

CSci 4011 INHERENT LIMITATIONS OF COMPUTER PROGRAMS

⋁ ⋁ ∧ ¬ A CIRCUIT x1 … is a collection of gates and inputs connected by wires. x2 x0 … is satisfiable if some setting of inputs makes it output 1. … can be encoded as a string CIRCUIT-SAT = {〈C〉 : C is a satisfiable circuit }

CIRCUIT-SAT IS DECIDABLE is_satisfiable(C): for each x ∈{0,1}n: if (circuitValue(C,x)==1): return True else: return False How long does the algorithm take on n-bit inputs? CIRCUIT-SAT ∈TIME(2O(n)) COMPLEXITY THEORY: What problems have efficient algorithms?

TIMECOMPLEXITY Definition: Let M be a TM that halts on all inputs. The running time or time-complexity of M is the function ƒ : N N, where ƒ(n) is the maximum number of steps that M uses on any input of length n. Definition: TIME(t(n)) = { L | L is a language decided by a O(t(n)) time Turing Machine }

FASTALGORITHMS Idea: a problem A is efficiently decidable if A  TIME( ) ?

COMPUTATIONALMODELS A = { 0k1k | k  0 }  TIME(n log n) No (single-tape) TM decides A in time o(n log n) A is decidable in time O(n) with a two-tape TM Theorem: Let t(n) be a function such that t(n)  n. Then every t(n)-time multi-tape TM has an equivalent O(t(n)2) single tape TM

 P = TIME(nc) c  ℕ IMPORTANTCOMPLEXITY CLASS P is the class of problems that can be solved in polynomial time: they are efficientlydecidable

WHY P? Many t(n) time variants of TMs can be simulated in time O(t(n)c) by a single-tape TM, including: TMs with double-unbounded tape: O(t(n)) Multi-tape TMs: O(t2(n)) RAM Machines: O(t6(n)) Polynomials are the simplest class of functions closed under composition, multiplication, etc… They “grow slowly” compared to exponentials

1 2 3 4 EXAMPLE: PATH A directed graph with n nodes… is a set of vertices V = {1,2,…,n} and a set of edges E⊆ V⨉V. can be encoded as an adjacency matrix 1, if (i,j) ∈E Aij = 0, otherwise has a path from s to t if there are nodes i1, …, ik such that { (s,i1), (i1, i2),…, (ik, t) } ⊆E. PATH = {〈G,s,t〉 |G has a path from s to t }

1 2 3 4 5 6 7 PATH IS IN P has_path(G,s,t): mark s and add it to the queue while the queue is not empty: take vi from the queue for each vj in {1,…, n}: if Aij = 1 and vj is not marked: mark vj and add it to the queue if t is marked: return True else: return False O(n) O(n) O(1) Total RAM Complexity: O(n2)

EXAMPLES Show that the following are in P: 3PATH = { 〈G,s,t〉 | There is a path of length at most 3 from s to t} EDFA = { 〈D〉| D is a DFA and L(D)=Ø } MEDIAN = { 〈(y1,…,yn),k〉 | median(y1,…,yn) is at least k }

STABLE MARRIAGES A High School has N boys and N girls. Each has a ranked list of dates for the 1951 Senior Prom. Bob Charlie Albert A,B,C A,C,B B,C,A B,A,C C,A,B C,B,A Betty Carol Alice An unstablecouple prefer each other to their current dates. STABLE = { 〈B,G〉 | There is a pairing with no unstable couple}

STABLE MARRIAGES in P Bob Charlie Albert A,B,C A,C,B B,C,A Alice C,A,B B,A,C C,B,A Betty Carol 1. Each boy asks his “first choice” to the prom. 2. Each girl “accepts” her best offer – for now. 3. Repeat until everyone has a date: a. Each boy with no date asks the next girl on his list. b. Each girl “accepts” her new best offer – for now. Total Time = O(n2)

NON-DETERMINISTIC TMs …are just like standard TMs, except: 1. The machine may proceed according to several possibilities. Formally, the transition function outputs a set of zero or more (state,symbol,direction) choices. 2. The machine accepts a string if there exists a path from start configuration to an accepting configuration 3. The running time of the machine is the length of the longest path to qaccept or qreject.

Deterministic Computation Non-Deterministic Computation reject accept or reject accept

{ L | L is decided by a O(t(n))-time non-deterministic Turing machine } Definition: NTIME(t(n)) = TIME(t(n))  NTIME(t(n)) NTIME(t(n)) ⊆ TIME(?)

Deterministic Computation Non-Deterministic Computation ntm_accepts(M,C): if (current_state(C) = qaccept): return True if (current_state(C) = qreject): return False for each C’∈M.(C): if ntm_accepts(M,C’): return True return False t(n) reject 2c accept NTIME(t(n))⊆TIME(2O(t(n)))

NON-DETERMINISTIC PROGRAMS …are just like standard programs, except: 1. There is a special instruction, guess(), that can return 0 or 1. 2. The program accepts an input if there exists a list of guesses that make it accept. 3. The running time of the program is the maximum number of steps that can be caused by calls to guess().

 NP = NTIME(nc) c  ℕ

logical operations parentheses variables BOOLEAN FORMULAS A satisfying assignment is a setting of the variables that makes the formula true (x  y)  z  = x = 1, y = 1, z = 1 is a satisfying assignment for  (x  y)  (z  x) 0 0 1 0

A Boolean formula is satisfiable if there exists a satisfying assignment for it a  b  c  d YES (x  y)  x NO SAT = {  |  is a satisfiable Boolean formula }

clauses A 3cnf-formula is of the form: (x1  x2 x3)  (x4  x2  x5) (x3  x2  x1) YES (x1 x̅2 x1) NO (x3  x1) (x3  x̅2 x̅1) NO (x1  x2  x3)  (x̅4 x2  x1) (x3  x1  x̅1) NO (x1  x̅2 x3) (x3  x̅2 x̅1) 3SAT = {  |  is a satisfiable3cnf-formula }

(   y  ) ( 0   y  0 ) 1 1 ( x   y  x ) ( 0   0  0 ) ( 0   1  0 ) 3SAT = {  |  is a satisfiable 3cnf-formula } Theorem: 3SAT  NP On input : 1. Check if the formula is in 3cnf 2. For each variable, non-deterministically substitute it with 0 or 1 3. Test if the assignment satisfies 

3SAT = {  |  is a satisfiable 3cnf-formula } Theorem: 3SAT  NP On input : 1. Check if the formula is in 3cnf 2. For each variable xi: a. Set temp = guess(). b. Scan across , replacing xi with temp. 3. Test if the assignment satisfies  Total running time = O(mn)

INHERENT LIMITATIONS OF COMPUTER PROGRAMS