THEORY OF COMPUTATION

1. THEORY OF COMPUTATION CSC 422 Evelyne Tropper

2. INTRODUCTION • Are all problems programmable? • What statement of a problem constitutes an implementable program? • Do the specifications of a program always lead to a program? • Is it always possible to find specifications of a problem that lead to a program? Evelyne Tropper

3. Purpose of Theory of Computation • Design a language for the mathematical specification of computer languages in general & for all computers • Describe the workings of a general computer in the simplest and most basic terms possible • Find a mathematical language acceptable to the above general, basic computer Evelyne Tropper

4. SETS & FUNCTIONS • { } A collection • {a,b,c} = A A set • CxS = {(c1,s1),…,(cn,sn)} Cartesian product • {0,1} Binary alphabet • e & |e|=0 Empty set & its length • wr Reverse of w (if w=wr ==> palindrome) • Ā Complement of A (A even nos ==> Ā odd nos)

5. RELATIONS • Reflexive if (a,a)  R  a  S • Symmetric if (a,b)  R ==> (b,a)  R • Transitive if (a,b)  R , (b,c)  R ==> (a,c)  R • Equivalence relation is reflexive, symmetric & transitive Evelyne Tropper

6. Examples of reflexivity • Let relation R be: a <= b then a <= a & R is reflexive • Let relation R be: a < b then a is not < a & R is not reflexive • Let set S be: {oaks}; let R be: oaks  trees then a  S => a  trees, so (a,a)  R Evelyne Tropper

7. Examples of symmetry • If R is (big,large) then a  {big} => a  {large} & a  {large} => a  {big} • If R is relation of synonyms then a synonym of b => b synonym of a so R is symmetric & also reflexive Evelyne Tropper

8. Examples of transitivity • If R is “on top of” & (a,b)  R; (b,c)  R then (a,c)  R & R is transitive • If R is “<” & (a,b)  R; (b,c)  R then (a,c)  R & R is transitive • If R is “child of” & (Bob, Mary)  R; (Mary, John) then R is not transitive • If R is “successor of” & (Bob, Mary)  R; (Mary, John) then R is transitive Evelyne Tropper

9. B universe A A A AB Ā A  B Union, Intersection & Complement Evelyne Tropper

10. Laws of Set Operations • Idempotency: AA = A, AA = A • Commutativity: AB=BA, AB=BA • Associativity: (AB)C= A(BC) (AB)C=A(BC) • Distributivity: (AB)C=(AC)(BC) (AB)  C=(AC) (BC) • Absorption: (AB)A=A, (AB)A=A • DeMorgan’s: A-(BC)=(A-B)(A-C) A-(BC)=(A-B)(A-C)

11. Examples • Apples  apples = apples • Apples  oranges = oranges  apples Dad  Mom = Mom  Dad … Evelyne Tropper

12. FUNCTIONS • R is a function if:  x  X ! y  Y : (x,y)  R • There are two notations: • f : X --> Y • f  X x Y • Alternate definition of a function:  x  X ! y  Y  f(x) = y Evelyne Tropper

13. X Y x y y1 For each x  X there is a unique y  Y In other words, the dotted line going from x to y1cannot exist Evelyne Tropper

14. X Y BIJECTIVE FUNCTION1-1 function • If y  Y ! x X then we have a Bijection or 1-1 function • If f is bijective =>  f--1(y) = x for f(x) = y x y x1 y1

15. FINITE AUTOMATA • Are all problems as easily programmable? • Consider a program to compute the integral of xn vs a sort program • The first has constant memory requirements , the second can be of arbitrary length • Can the implementation of a program be designed as a program? input output program

16. Deterministic Finite Automata • A DFA is a program to design programs that use a constant amount of memory. • A DFA can be thought of as a tape which reads one character at the time until the end of the tape. Each position on the tape puts DFA in a different state. • It can elucidate errors that may show up in a programming problem. Evelyne Tropper

17. Example 1 When a customer pays for goods with electronic money, that is with an ATM card, all possibilities must be accounted for : • The customer may decide to pay, i.e. sends the money to the store • The customer may cancel & money is sent to bank to be deposited in customer’s account • The store may ship the goods to the customer • The store may redeem the money, i.e. the money is sent to the bank to be given to the store • The bank may send the money to the store Can the store ship the goods without ever getting paid? Evelyne Tropper

18. Start pay redeem transfer d 4 3 a e f b ship ship ship redeem transfer c g 2 cancel Redeem transfer 1 cancel start pay DFA for previous problem For the store For the bank Forthe customer

19. 1/1 0/0 1/0 q0 q1 0/1 Example 2Memory machine

20. States Outputs Input q q q q Time 0 1 0 1 0 q q 0 1 t 0 1 1 q q 1 0 t+1 1 0 0/1 0/0 1/1 q0 q1 1/0 Example 3Parity machine

21. 10/0 10/1 11/0 000 01/1 01/0 carry 00/1 00/0 11/1 Example 4Adding machine 1 1 1 101101 111001 1100110 States are: ( i1 i2 carry) (000), (001), (010), (011), (100), (101), (110), (111)

22. Definition of a DFA DFA A = (Q, , , s, F) Q - finite set of states  - finite input alphabet  - transition function from Q x  --> Q s - initial state s  Q F - favorable, or accepting, states F  Q Evelyne Tropper

23. Applications of a DFA • Search engines on Web can use them • News analysts searching on-line for special topics • Stock analysts searching for stock names • “Shopping robots” searching for best price on-line • Searching for all books on Amazon containing a certain phrase or word • grep, egrep, fgrep in Unix Evelyne Tropper

24. Functioning of a DFA • Automaton A in state q  Q • reads a   (letter in the alphabet) • enters stateq1 =  (q, a) which is determined completely by current state & which is the content of the current cell. • Set of all accepted input words by A is called the language L(A) of the DFA. Evelyne Tropper

25. Initial state is preceded by Finite State Diagram It is a directed graph representing a DFA. a State q changing to state q1 after reading input a q q1 A favorable state is doubly circled. Evelyne Tropper

26. Statmt 1 If-then Program A statement 1 if ---- then exit else ---- end statement 2 statement 3 end s0 exit s1 end s2 s3 Statmt 2 s5 end s4 end Statmt 3 Programs & Finite State Diagrams If-else Evelyne Tropper

27. Example 1 a b a s r q b a b Accepts language, L(anbm) Ex: statement, statement,…, if-then, if-then, …, end

28. Not accepted Trap stays there Example 2 b a a s q r a b b Accepts any language with 2 consecutive a’s L(…, a, a, …) Ex: Any program with a nested pair of “while” loops

29. Dead, no further state Example 3 b a a s q r a b b It will not accept two consecutive a’s L(…, a, a, …) (Ex: no nested “for” loops) It is the complement of the previous example

30. s q a b a f a b Example 4 b • Initial “a” must be followed by “a” to be accepted • Initial “b” can be followed by any number of a’s & b’s

31. Example 5 b b a a s f q a b L(a, bn, a, bm) a b p Ex: one “case” followed by multiple statements Evelyne Tropper

32. Configurations • In chess there are individual moves & standard patterns for the opening or end game. • Imagine that the players have reached the end game; the pieces left are BK, BQ, 1 BR, 1 BKt, 3 P; WK,WQ, 1 WB, 1 WKt, 2 P. • Imagine they are in a certain configuration, that is a certain pattern on the board. • Then the next moves can be figured out from that point on. Evelyne Tropper

33. Definition of a configuration A configuration is a composite of state (pieces left), position (pattern on the board), input (next moves). Ex: In reading input {aaaabba}, after state {aaa} has been reached, position q can be reached by reading input {abba} (q,abba) is a configuration towards acceptance. w is accepted if it yields a favorable state. If (q,w) -->(q1,w1) then  σ   : w = σ w1 & δ (q, σ ) = q1 then(q,w) yields (q1,w1) inone step (q,w) yields (q1,w1) if  a sequence of configurations: (q1,w1) … (qk,wk) : (q1,w1) = (q,w) , (qk,wk) = (q1,w1) & (qi,wi) yields (qi+1,wi+1) in one step. All configurations yield unique configurations as it is deterministic.

34. 10c 10c 10c 10c 5 a s 20 15 10 5c 5c 5c 5c 5c 25c Example 1 A vending machine for newspapers. The cover is released when $0.25 is reached. It does not return money if > $0.25 is put in.  = ( 5c, 10c, 25c ) Evelyne Tropper

35. Examples of applications in Computer Science • Programming sequences, branching, loops • Pattern matching (for WWW & AI) • Lexical analysis in compilers • Finite state machines in software specs & design • Word processors • Design of telecommunication protocols • Design of circuits for VLSI • Hardware design • Control mechanisms Evelyne Tropper

36. Non-deterministic Finite Automata • Union of 2 DFAs gives a NDFA as more than 1 arrow from any state with the same input. • For example, in pattern matching of A  B, we may match A or we may match B. • NDFA A = ( Q,,,s,F ) where   Q x (  {e}) x Q is a transition relation (q, a, p)   The empty string as input permits the automaton to jump from one state to the other. So after A is matched, whatever state the other branch is at, can jump

37. Example • The ability to be in several different states at once, can be expressed as the ability to “guess” what will come next. • When a system searches for a certain sequence of characters, such as a keyword, we can use a sequence of states to do nothing but jump from state to state until we find the keyword. Evelyne Tropper

38. e b w i n t e r e t n Example 1Find first occurrence of keywords Searching for the keywords: “Web” or “Internet” Finding either gets us to a favorable state. Evelyne Tropper

39. a b q q1 q2 a b Functioning of a NDFA vs DFA • Automaton A in state q  Q • reads a   (letter in the alphabet) • enters state q1or q2 =  (q, a) which is determined completely by current state & which is the content of the current cell. State q changing to state q1 or state q2 after reading input a b Evelyne Tropper

40. Interpretation • One input can send you to 2 different sub-programs (matching 2 Jones in a DB) • This is useful when you build a program from multiple sub-programs. In industry, whole teams work on complicated programs, each member on separate sub-programs. • In parallel programming, this represents different threads. • NDFAs are easier to design & can be changed to DFAs Evelyne Tropper

41. q e s e p b a b b s q r b Examples a No jumping from q to p Therefore no need for trap state (q,w) can yield many different configurations (q1, e) Automaton that begins & ends with b

42. Definition & theorem • Definition: A & A1 that accept the same language are equivalent. • Theorem: For all non-deterministic automaton A there exists a deterministic finite automaton A1 equivalent to A. Evelyne Tropper

43. a b a b {t}---> {t} {s}---> {t} {s}---> {r,t} {s}---> {q,r,t} Example e q r This NDFA is equivalent to the following deterministic transitions: e a b a b t s b Accepting states are: {t}; {r,t}; {q,r,t} Trap state is {0}

44. {t} {q,r,t} {r,t} Equivalent Deterministic Automaton b a {s} b a Evelyne Tropper

45. Regular Expressions Regular expressions are used to: • Design a language for mathematical specification of languages acceptable by a finite automata (computers of all types) • Do that by a general algorithmic procedure • To convert FA back to its specs through an algorithmic procedure. • These are used in turn to do simulations. Evelyne Tropper

46. Reg. Exp. vs FA • DFA & NDFA are machine-like descriptions • Regular Expressions are algebraic-like descriptions. Evelyne Tropper

47. Applications of Regular Expressions • Lexical analyzers such as Lex & Flex which take source code and convert it into tokens • Grep in Unix • Alternative to FA notation for describing software components • A declarative way to express which strings are acceptable Evelyne Tropper

48. Language for regular expressions Some of the complex programming languages can be obtained from simpler languages using , , , concatenation & Kleene stars. Concatenation: of strings u, v is “uv” of languages L1, L2 is L1L2 = {uv | u L1, v  L2} Kleene Star: L* of L is the infinite union: {e}  L  L2  L3 … L* = {w1w2 …wk} when wi  L Evelyne Tropper

49. Examples L  M = {001, 10, 111}  {e, 001} = {e, 001, 10, 111} L  M = {001} LM = {001, 10, 111, 001001, 10001, 111001} L* for {0, 1} = all strings of 0’s & 1’s for {0, 11} = all strings of 0’s & 1’s such that 1’s come in pairs L0 = {e}; L1 = {0, 11}; L2 = {00, 011, 110, 1111}; L3 = {000, 0011, 0110, 01111, 1100, 11011, 11110, 111111} … Evelyne Tropper

50. . . . . . . s s Languages accepted by FA Theorem: If languages L & M are acceptable by a finite automaton, so are L  M, L  M, * - L (complement), L – M, LM (conc), L* (Kleene Star) Suppose L accepted by Suppose M accepted by Evelyne Tropper