CSCI 3130: Formal languages and automata theory

1. Fall 2010 The Chinese University of Hong Kong CSCI 3130: Formal languages and automata theory Polynomial time Andrej Bogdanov http://www.cse.cuhk.edu.hk/~andrejb/csc3130

2. Running time When you do laundry, you want to know: How soon will the laundry be done? When you run a program, you want to know: M 01001 How soon will my program finish?

3. Efficiency • Undecidable problems: We cannot find solutions in any finite amount of time • Decidable problems: Sometimes we can find solutions fast, sometimes not PCP ATM decidable efficient

4. Efficiency • The running time of an algorithm depends on the input • For longer inputs, we should allow more time • Efficiency is measured as a function of input size PCP ATM decidable efficient

5. Running time • The running time of TM M is the function tM(n): tM(n) = maximum number of steps that M takes on any input of length n L = {w#w: w∈ {a, b}} O(n) times M: On input x, until you reach # O(n) steps Read and cross off first a or b before # Read and cross off first a or b after # If there is a mismatch, reject O(n) steps If all symbols but # are crossed off, accept running time: O(n2)

6. Running time L = {0n1n: n ≥ 0} O(n) steps O(n) times M: On input x, Check that the input is of the form 0*1* Until everything is crossed off: Move head to left end of tape Cross off the leftmost 0 Cross off the following 1 If everything is crossed off, accept. O(n) steps running time: O(n2)

7. A faster way L = {0n1n: n ≥ 0} O(n) steps O(log n) times M: On input x, Check that the input is of the form $0*1* Until everything is crossed off: Move head to left end of tape Find the parities of number of 0s and 1s If one is even and other off, reject Otherwise, cross off every other 0 and every other 1 If everything is crossed off, accept. O(n) steps O(n) steps running time: O(n log n)

8. Running time vs. model • What if we have a two-tape Turing Machine? L = {0n1n: n ≥ 0} O(n) steps M: On input x, Check that the input is of the form 0*1* Copy 0* part of input on second tape Until ☐ is reached, Cross off next 1 from first tape and next 0 from second tape If both tapes reach ☐ at same time, accept O(n) steps running time: O(n)

9. Running time vs. model • How about a java program? L = {$0n1n: n ≥ 0} M(string x) { n = x.len; if n % 2 == 0 reject; else for (i = 1; i <= n/2; i++) if x[i] != 0 reject; if x[n-i+1] != 1 reject; accept; } 1-tape TM 2-tape TM java running time: O(n) The running time can change depending on the model of computation! O(n log n) O(n) O(n)

10. Measuring running time • What does it mean when we say: • One “time unit” inall mean different things! “This algorithm runs in time T” java RAM machine 1-tape TM d(q3, a) = (q7, b, R) if (x > 0) y = 5*y + x; write r3;

11. Efficiency and the Church-Turing thesis • The Church-Turing thesis says all these models are equivalent in power… … but not in running time! java Turing Machine RAM machine multitape TM

12. The Cobham-Edmonds thesis • However, there is an extension to the Church-Turing thesis that saysFor any realistic models of computation M1 and M2: • So any task that takes time T on M1 can be done in time (say) O(T2) or O(T3) on M2 M1 can be simulated on M2 with at mostpolynomial slowdown

13. Efficient simulation • The running time of a program depends on the model of computation…… but in the grand scheme, this is irrelevant java ordinary TM multitape TM RAM machine slow fast Every reasonable model of computation can be simulated efficiently on every other

14. Example of efficient simulation • Recall simulating multiple tapes on a single tape … … M G = {0, 1, ☐} … 1 0 0 0 1 1 0 0 S …    # 0 1 0 # 0 1 # 1 0 0 # G’ = {0, 1, ☐, 0, 1, ☐, #}   

15. Running time of simulation • Each move of the multiple tape TM might require traversing the whole single tape 1 step of 3-tape TM O(s) steps of single tape TM s = rightmost cell ever visited s ≤ 3t + 4 after t steps t steps of 3-tape O(ts) = O(t2) single tape steps quadraticslowdown multi-tape TM single tape TM

16. Simulation slowdown • Cobham-Edmonds Thesis: java multi-tape TM O(t) O(t) O(t2) O(t) O(t2) O(t) RAM machine single tape TM M1 can be simulated on M2 with at mostpolynomial slowdown

17. The class P decidable P is the class of languages that can be decided on a TM with polynomial running time in the input length efficient context-free By the CE thesis, we can replace “ordinary TM” by any realisticmodel of computation regular java RAM multi-tape TM

18. Examples of languages in P P is the class of languages that are decidable in polynomial time (in the input length) L01 = {0n1n: n > 0} decidable LG = {x: x is generated by G} G is some CFG P (efficient) PATH = {(G, s, t): G is a graph with a pathfrom node s to node t} PATH LG context-free L01

19. Context-free languages in polynomial time • Let L be a context-free language, and G be a CFG for L in Chomsky Normal Form CYK algorithm • For cells in last rowIf there is a production A  xiPutA in table cell ii • For cells st in other rows If there is a production A  BC whereB is in cell sj and C is in cell jtPutA in cell st 1k … … 23 12 22 nn 11 x1 x2 … xn On input x of length n, running time is O(n3) ✔

20. Paths in polynomial-time x s PATH = {〈G, s, t〉: G is a graph with a pathfrom node s to node t} x x G has n vertices, m edges x M := On input 〈G, s, t〉, where G is a graph with nodes s and t t x Place a mark on node s. O(n) times Repeat until no additional nodes are marked: Scan the edges of G. If there is an edge so that a is marked and b is not marked, mark b. O(m) time If tis marked accept, otherwise reject. ✔ running time: O(nm)

21. Hamiltonian paths • A Hamiltonian path in G is a path that visits every node exactly once s t UHAMPATH = {〈G, s, t〉: G is a graph with a Hamiltonian pathfrom s to t} We don’t know if UHAMPATH is in P, and we believe it is not