Algorithmic problems l.jpg
This presentation is the property of its rightful owner.
Sponsored Links
1 / 42

Algorithmic Problems PowerPoint PPT Presentation


  • 103 Views
  • Uploaded on
  • Presentation posted in: General

Algorithmic Problems. Zeph Grunschlag. Agenda. Viewing algorithms as languages Algorithmic Decision Problems A TM , A CFG , A REX , E TM , … etc. Knowing the Unknowable. Arrived at important juncture in course. Now we will study one of the fundamental questions in Computer Science:

Download Presentation

Algorithmic Problems

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Algorithmic problems l.jpg

Algorithmic Problems

Zeph Grunschlag


Agenda l.jpg

Agenda

  • Viewing algorithms as languages

  • Algorithmic Decision Problems

    • ATM, ACFG , AREX, ETM, … etc.


Knowing the unknowable l.jpg

Knowing the Unknowable

Arrived at important juncture in course. Now we will study one of the fundamental questions in Computer Science:

What is the ultimate power of computational machinery?

Of equal importance in human quest to understand self:

What are the impenetrable limits of human knowledge?


Knowing the unknowable4 l.jpg

Knowing the Unknowable

Accepting the Church-Turing thesis implies that these are the same question! Church-Turing thesis is usually recognized as applying to human computations, and therefore human thought processes.

Amazing Fact: It is possible to give precise examples of problems which are beyond the reach of mechanical computation as well as human intellect. We in fact “know the unknowable.”


Decision problems l.jpg

Decision Problems

A decision problem is a set of yes/no questions, each called an instance. For the decision problem to be interesting, the set of questions should be infinite.

EG: “Does z equal the sum of x and y ?” is a good example. A computer program that can give the correct solution for any possible x,y,z intuitively must have figured out how to add numbers.

EG: “Does 8 equal the sum of 3 and 5?” is a bad example. Could easily write program for this instance without knowing how to add!


Decision problems6 l.jpg

Decision Problems

Q: How about the problem “Does God Exist?” Can a computer solve this problem?


Decision problems7 l.jpg

Decision Problems

A: In the way defined above, the answer is yes! We give two computer programs, both of which accept the input “Does God exist?”

One of these programs is correct! Therefore, that computer program has solved the problem of deciding if God exists!

Q: Any misgivings?

A(String question){

return “No!”

}

B(String question){

return “Yes!”

}


Decision problems8 l.jpg

Decision Problems

A: Many possible misgivings. Here’s one:

“Solution” to existence of God surely not what is meant by solving the problem. We expected the solution to be arrived at by careful consideration of physical evidence, or by theological arguments, or some other intricate process.

CS viewpoint: Any algorithm which works only on a finite set of inputs is considered meaningless, as could always program a look-up table which gave correct solutions without using any insight.


Decision problems9 l.jpg

Decision Problems

So to solve the deity existence problem would have to give a more general solution. EG:

Given: A complete description of a physical universe and a complete description of a deity.

Decide: Does the deity necessarily exist for the given universe?

Now there are infinitely many possible inputs, so cannot just use a look-up table and this is a meaningful problem, though hopeless…


Decision problems as languages l.jpg

Decision Problems as Languages

To study what it means for a decision problem to be solvable, need a method of standardizing such problems and making them easily understood by computers. Can’t just come over to a computer and ask “Does 2 + 2 = 4?” and even if could, dealing with voice recognition, natural language processing, etc. takes us far afield to topics irrelevant to answering “is the problem solvable?”

Solution: represent each instance by a string:

Thus instance is the string “Does 2 + 2 = 4?” or more simply “2+2=4”.

Q: Define solvability language theoretically?


Decision problems as languages11 l.jpg

Decision Problems as Languages

A: A decision problem P is solvable if its encoding by strings admits a computer program which inputs such encodings, always halts, and always outputs the correct answer. Furthermore, the set of encodings should be computable in its own right.

Language theoretically: Let S be the alphabet in which the encoded instances of P exist. Let E be the set of encoded instance and let L be the set corresponding to the YES instances.1 Then P is said to be decidable (or algorithmically solvable or computable) if L is decidable.


Decision problems sensible encodings l.jpg

Decision ProblemsSensible Encodings

Some common sense is needed: The encoding scheme should be sensible but there is no way to define what sensible means.

EG: “Is p a prime number?”

  • Encoding scheme which results in the positive instances L = {2,3,5,7,…} is sensible.

  • Encoding scheme which results in the positive instances L = {1,4,6,8,9,10,…} is not!


Decision problems sensible encodings13 l.jpg

Decision ProblemsSensible Encodings

Sensible encoding of sensible decision problem would never result in an undecidable encoding set E but condition still needed so don’t solve impossible problems unwittingly.

EG: “Does a given Java program with no infinite loops print out “I’m not loopy!” on some input?”

Natural encoding set E is over Unicode alphabet, and is just the set of Java programs with no infinite loops. But we’ll see that E is itself undecidable so second part becomes meaningless.


Decision problems basic problems l.jpg

Decision ProblemsBasic Problems

Before tackling fundamental decision problems of CS, start introspectively by considering if there are algorithms for…

  • …telling if a given string is accepted by a given language of a fixed type C such as DFA’s, NFA’s, PDA’s etc. Acceptance ProblemAC

  • …telling if a given language of type is C empty. Emptiness ProblemEC

  • …telling if two given languages of the same type C are equal. Equivalence ProblemEQC

    Each in turn, is encoded by a language.


Decision problems encoding example l.jpg

Decision ProblemsEncoding Example

Let’s see how ADFA may be encoded.

ADFA is the problem of deciding

Given: A DFA M and a string x.

Decide: Does M accept x ?

The language L of interest is the language of encodings of pairs (M, x ) where M accepts x. Encoding is denoted using the angled braces notation “<M, x >”. In practice, will never really work with encodings as too unwieldy. However, for once it’s worth to see how this might actually be carried out.


Decision problems encoding example16 l.jpg

Decision ProblemsEncoding Example

a

Consider the DFA:

Q: How can you represent this by a string?

0,1

0,1

b


Decision problems encoding example17 l.jpg

Decision ProblemsEncoding Example

a

A: DFA’s are 5-tuples,

so just write down the

5 tuple, spelling out all

details. Therefore

<M > =

({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a})

0,1

0,1

b


Decision problems encoding example18 l.jpg

Decision ProblemsEncoding Example

a

A: DFA’s are 5-tuples,

so just write down the

5 tuple, spelling out all

details. Therefore

<M > =

({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a})

Q

0,1

0,1

b


Decision problems encoding example19 l.jpg

Decision ProblemsEncoding Example

a

A: DFA’s are 5-tuples,

so just write down the

5 tuple, spelling out all

details. Therefore

<M > =

({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a})

S

0,1

0,1

b


Decision problems encoding example20 l.jpg

Decision ProblemsEncoding Example

a

A: DFA’s are 5-tuples,

so just write down the

5 tuple, spelling out all

details. Therefore

<M > =

({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a})

d

0,1

0,1

b


Decision problems encoding example21 l.jpg

Decision ProblemsEncoding Example

a

A: DFA’s are 5-tuples,

so just write down the

5 tuple, spelling out all

details. Therefore

<M > =

({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a})

q0

0,1

0,1

b


Decision problems encoding example22 l.jpg

Decision ProblemsEncoding Example

a

A: DFA’s are 5-tuples,

so just write down the

5 tuple, spelling out all

details. Therefore

<M > =

({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a})

F

0,1

0,1

b


Decision problems encoding example23 l.jpg

Decision ProblemsEncoding Example

a

A: DFA’s are 5-tuples,

so just write down the

5 tuple, spelling out all

details. Therefore

<M > =

({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a})

Encoding <M,w > amounts to just adding the comma and the input string w.

Q: What’s the alphabet of encoding lang. E ?

0,1

0,1

b


Decision problems encoding example24 l.jpg

Decision ProblemsEncoding Example

A: TOO BIG! The number of letters in alphabet and number of states is unbounded. Using different characters for each state/letter would require an infinite alphabet if want to be able to encode every possible DFA! Instead, should fix some alphabet (e.g. binary or ASCII) and refer to states/letters using numbers. Thus the state a is denoted by 1, since it’s the first state, the state b by 2 and so on…


Decision problems what you really do l.jpg

Decision ProblemsWhat You Really Do!

After going to all this effort to define decision problems in terms of languages, Turing machines, encodings and so on, in practice, we use none of the above. The point was to argue that in principle this could be done, and remind ourselves that this ought to be achievable, when wondering whether a not purported pseudocode is really an algorithm.

Sipser’s approach tries to maintain the veneer of this approach. I find this to be artificial at times so will just work directly with the natural data structure inherited by the problem.


A dfa l.jpg

ADFA

For example, the easiest way to view a DFA is as a labeled graph. Encoding the DFA by a string and trying to apply some graph theoretic algorithm to the representative string is overly burdensome. You basically have to recreate the whole graph on the TM tape, which in principle can be done, but in practice is silly, especially since the starting point was a graph!

Q: How do you solve ADFA ?


A dfa27 l.jpg

ADFA

A: Just follow the path labeled by the input string from the start state. Accept if end was in F. More precisely.

DFAaccept(DFA M, String x1 x2 x3 … xn )

State q = q0//q0 defined by M’s 5-tuple

for(i = 1 to n)

q = d(q,xi)// d defined by M’s 5-tuple

if(q  F)// F defined by M’s 5-tuple

return ACCEPT

else

return REJECT


A nfa l.jpg

ANFA

Q: How about ANFA?


A nfa29 l.jpg

ANFA

A: Just determinize first and use solution to ADFA.

NFAaccept(NFA N, String x1 x2 x3 … xn )

DFA M = determinize(N)//Subset Constr’n

return DFAaccept(M, x1 x2 x3 … xn )

Q1: What’s the running time?

Q2: What’s not ideal about this algorithm?

Q3: What’s the fix?


A nfa30 l.jpg

ANFA

A1: O(2|N|+n) since determinizing might involve all the subsets of the states of N so is exponential, but once have determinized, simulating the DFA is O(n).

Q2: The problem is the exponential running time.

Q3: We already know the fix. It’s just the game “Determinize” which keeps track of active state without explicitly constructing all the subsets. Running time of O(|N |2 ·n) since each update of the active states may involve looking at O(|N |) active states each of which may activate O(|N |) other states.


A nfa31 l.jpg

ANFA

NFAaccept2(NFA N, String x1 x2 x3 … xn )

StateSet S = {q0 }// store S as bit string

for(i = 1 to n)

S = d(S,xi)// since NFA, d outputs a set

if(S and F are not disjoint)

return ACCEPT

else

return REJECT


E dfa l.jpg

EDFA

Let’s now focus on emptiness problem. We can use the pumping lemma to solve the emptiness problem: Suppose we know what the pumping number p is for our DFA. Then if we found no accepted strings of length < p we could be sure that the DFA accepts no strings.

Q: Why is this true?


E dfa33 l.jpg

EDFA

A: Suppose language not empty. Since checked that all strings of length < p are rejected, smallest accepted string s must be of length  p. The string s is pumpable, so can be pumped down obtaining a shorter string s’, which contradicts the minimality of s !

This prompts the following algorithm:


E dfa34 l.jpg

EDFA

DFAempty( DFA M )

integer p = |Q |

for (all strings x over S with length < p)

if( DFAaccepts(M,x) == ACCEPT )

return NONEMPTY

//only got here if nothing was accepted

return EMPTY

Q: Running time? Improvements?


E dfa35 l.jpg

EDFA

A:

Running time: O(|S||Q|) –exponential.

Improvements: Just see if any accept state can be reached from start state by performing a BFS or DFS as follows:


E dfa36 l.jpg

EDFA

DFAempty2( DFA M )

State q = q0

Stack S // Initialize a LIFO stack

Set V =// Set of visited states

S.push(q0)

while(S  )

q = S.pop()

if (qF ) return NONEMPTY

V = V  {q}

for (States q’ satisfying qq’ ) // an edge

if (q’ V ) S.push(q’ )

return EMPTY // Only got here if F unreachable

Q: Is this a BFS or DFS?


E dfa37 l.jpg

EDFA

A: This is a DFS algorithm since stack is used. Queue would give BFS.


A cfg l.jpg

ACFG

Acceptance problem is also solvable. There’s a rather horrible algorithm involving Chomsky Normal Form. Later we’ll see a much improved polynomial time algorithm (must have one, otherwise compilers would be useless!)

Chomsky normal form gives the following fact:

LEMMA: Suppose a grammar G is in Chomsky normal form. Let x be in L(G ), and let n be the length of x. Then x is generated by a derivation of length 2n-1, when n > 0.


A cfg39 l.jpg

ACFG

LEMMA: Suppose a grammar G is in Chomsky normal form. Let x be in L(G ), and let n be the length of x. Then x is generated by a derivation of length 2n-1, when n > 0.

Proof. The first n-1 productions are of the form A BC and get us to the correct length. The last n production are unit terminating A  a and derive x.

This gives rise to the following algorithm:


A cfg40 l.jpg

ACFG

CFGaccept( CFG G, String x=x1 x2 x3 … xn )

CFG G’ = ChomskyNormalForm(G )

for (all derivations from start variable

of G’ of length  2n +1)

if (derivation resulted in x)

return ACCEPT

return REJECT

Q1: Why does this work for x =e?

Q2: Running time in terms of n ?


A cfg41 l.jpg

ACFG

A1: Works for x =e because of “ 2n+1” clause.

A2: Exponential. If we consider the blow-up of CNF conversion, this is a truly horrendous algorithm.


Blackboard exercises l.jpg

Blackboard Exercises

  • EQDFA

  • ECFG

  • CARDDFA


  • Login