- 97 Views
- Uploaded on
- Presentation posted in: General

Algorithmic Problems

Algorithmic Problems

Zeph Grunschlag

- Viewing algorithms as languages
- Algorithmic Decision Problems
- ATM, ACFG , AREX, ETM, … etc.

Arrived at important juncture in course. Now we will study one of the fundamental questions in Computer Science:

What is the ultimate power of computational machinery?

Of equal importance in human quest to understand self:

What are the impenetrable limits of human knowledge?

Accepting the Church-Turing thesis implies that these are the same question! Church-Turing thesis is usually recognized as applying to human computations, and therefore human thought processes.

Amazing Fact: It is possible to give precise examples of problems which are beyond the reach of mechanical computation as well as human intellect. We in fact “know the unknowable.”

A decision problem is a set of yes/no questions, each called an instance. For the decision problem to be interesting, the set of questions should be infinite.

EG: “Does z equal the sum of x and y ?” is a good example. A computer program that can give the correct solution for any possible x,y,z intuitively must have figured out how to add numbers.

EG: “Does 8 equal the sum of 3 and 5?” is a bad example. Could easily write program for this instance without knowing how to add!

Q: How about the problem “Does God Exist?” Can a computer solve this problem?

A: In the way defined above, the answer is yes! We give two computer programs, both of which accept the input “Does God exist?”

One of these programs is correct! Therefore, that computer program has solved the problem of deciding if God exists!

Q: Any misgivings?

A(String question){

return “No!”

}

B(String question){

return “Yes!”

}

A: Many possible misgivings. Here’s one:

“Solution” to existence of God surely not what is meant by solving the problem. We expected the solution to be arrived at by careful consideration of physical evidence, or by theological arguments, or some other intricate process.

CS viewpoint: Any algorithm which works only on a finite set of inputs is considered meaningless, as could always program a look-up table which gave correct solutions without using any insight.

So to solve the deity existence problem would have to give a more general solution. EG:

Given: A complete description of a physical universe and a complete description of a deity.

Decide: Does the deity necessarily exist for the given universe?

Now there are infinitely many possible inputs, so cannot just use a look-up table and this is a meaningful problem, though hopeless…

To study what it means for a decision problem to be solvable, need a method of standardizing such problems and making them easily understood by computers. Can’t just come over to a computer and ask “Does 2 + 2 = 4?” and even if could, dealing with voice recognition, natural language processing, etc. takes us far afield to topics irrelevant to answering “is the problem solvable?”

Solution: represent each instance by a string:

Thus instance is the string “Does 2 + 2 = 4?” or more simply “2+2=4”.

Q: Define solvability language theoretically?

A: A decision problem P is solvable if its encoding by strings admits a computer program which inputs such encodings, always halts, and always outputs the correct answer. Furthermore, the set of encodings should be computable in its own right.

Language theoretically: Let S be the alphabet in which the encoded instances of P exist. Let E be the set of encoded instance and let L be the set corresponding to the YES instances.1 Then P is said to be decidable (or algorithmically solvable or computable) if L is decidable.

Some common sense is needed: The encoding scheme should be sensible but there is no way to define what sensible means.

EG: “Is p a prime number?”

- Encoding scheme which results in the positive instances L = {2,3,5,7,…} is sensible.
- Encoding scheme which results in the positive instances L = {1,4,6,8,9,10,…} is not!

Sensible encoding of sensible decision problem would never result in an undecidable encoding set E but condition still needed so don’t solve impossible problems unwittingly.

EG: “Does a given Java program with no infinite loops print out “I’m not loopy!” on some input?”

Natural encoding set E is over Unicode alphabet, and is just the set of Java programs with no infinite loops. But we’ll see that E is itself undecidable so second part becomes meaningless.

Before tackling fundamental decision problems of CS, start introspectively by considering if there are algorithms for…

- …telling if a given string is accepted by a given language of a fixed type C such as DFA’s, NFA’s, PDA’s etc. Acceptance ProblemAC
- …telling if a given language of type is C empty. Emptiness ProblemEC
- …telling if two given languages of the same type C are equal. Equivalence ProblemEQC
Each in turn, is encoded by a language.

Let’s see how ADFA may be encoded.

ADFA is the problem of deciding

Given: A DFA M and a string x.

Decide: Does M accept x ?

The language L of interest is the language of encodings of pairs (M, x ) where M accepts x. Encoding is denoted using the angled braces notation “<M, x >”. In practice, will never really work with encodings as too unwieldy. However, for once it’s worth to see how this might actually be carried out.

a

Consider the DFA:

Q: How can you represent this by a string?

0,1

0,1

b

a

A: DFA’s are 5-tuples,

so just write down the

5 tuple, spelling out all

details. Therefore

<M > =

({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a})

0,1

0,1

b

a

A: DFA’s are 5-tuples,

so just write down the

5 tuple, spelling out all

details. Therefore

<M > =

({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a})

Q

0,1

0,1

b

a

A: DFA’s are 5-tuples,

so just write down the

5 tuple, spelling out all

details. Therefore

<M > =

({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a})

S

0,1

0,1

b

a

A: DFA’s are 5-tuples,

so just write down the

5 tuple, spelling out all

details. Therefore

<M > =

({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a})

d

0,1

0,1

b

a

A: DFA’s are 5-tuples,

so just write down the

5 tuple, spelling out all

details. Therefore

<M > =

({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a})

q0

0,1

0,1

b

a

A: DFA’s are 5-tuples,

so just write down the

5 tuple, spelling out all

details. Therefore

<M > =

({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a})

F

0,1

0,1

b

a

A: DFA’s are 5-tuples,

so just write down the

5 tuple, spelling out all

details. Therefore

<M > =

({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a})

Encoding <M,w > amounts to just adding the comma and the input string w.

Q: What’s the alphabet of encoding lang. E ?

0,1

0,1

b

A: TOO BIG! The number of letters in alphabet and number of states is unbounded. Using different characters for each state/letter would require an infinite alphabet if want to be able to encode every possible DFA! Instead, should fix some alphabet (e.g. binary or ASCII) and refer to states/letters using numbers. Thus the state a is denoted by 1, since it’s the first state, the state b by 2 and so on…

After going to all this effort to define decision problems in terms of languages, Turing machines, encodings and so on, in practice, we use none of the above. The point was to argue that in principle this could be done, and remind ourselves that this ought to be achievable, when wondering whether a not purported pseudocode is really an algorithm.

Sipser’s approach tries to maintain the veneer of this approach. I find this to be artificial at times so will just work directly with the natural data structure inherited by the problem.

For example, the easiest way to view a DFA is as a labeled graph. Encoding the DFA by a string and trying to apply some graph theoretic algorithm to the representative string is overly burdensome. You basically have to recreate the whole graph on the TM tape, which in principle can be done, but in practice is silly, especially since the starting point was a graph!

Q: How do you solve ADFA ?

A: Just follow the path labeled by the input string from the start state. Accept if end was in F. More precisely.

DFAaccept(DFA M, String x1 x2 x3 … xn )

State q = q0//q0 defined by M’s 5-tuple

for(i = 1 to n)

q = d(q,xi)// d defined by M’s 5-tuple

if(q F)// F defined by M’s 5-tuple

return ACCEPT

else

return REJECT

Q: How about ANFA?

A: Just determinize first and use solution to ADFA.

NFAaccept(NFA N, String x1 x2 x3 … xn )

DFA M = determinize(N)//Subset Constr’n

return DFAaccept(M, x1 x2 x3 … xn )

Q1: What’s the running time?

Q2: What’s not ideal about this algorithm?

Q3: What’s the fix?

A1: O(2|N|+n) since determinizing might involve all the subsets of the states of N so is exponential, but once have determinized, simulating the DFA is O(n).

Q2: The problem is the exponential running time.

Q3: We already know the fix. It’s just the game “Determinize” which keeps track of active state without explicitly constructing all the subsets. Running time of O(|N |2 ·n) since each update of the active states may involve looking at O(|N |) active states each of which may activate O(|N |) other states.

NFAaccept2(NFA N, String x1 x2 x3 … xn )

StateSet S = {q0 }// store S as bit string

for(i = 1 to n)

S = d(S,xi)// since NFA, d outputs a set

if(S and F are not disjoint)

return ACCEPT

else

return REJECT

Let’s now focus on emptiness problem. We can use the pumping lemma to solve the emptiness problem: Suppose we know what the pumping number p is for our DFA. Then if we found no accepted strings of length < p we could be sure that the DFA accepts no strings.

Q: Why is this true?

A: Suppose language not empty. Since checked that all strings of length < p are rejected, smallest accepted string s must be of length p. The string s is pumpable, so can be pumped down obtaining a shorter string s’, which contradicts the minimality of s !

This prompts the following algorithm:

DFAempty( DFA M )

integer p = |Q |

for (all strings x over S with length < p)

if( DFAaccepts(M,x) == ACCEPT )

return NONEMPTY

//only got here if nothing was accepted

return EMPTY

Q: Running time? Improvements?

A:

Running time: O(|S||Q|) –exponential.

Improvements: Just see if any accept state can be reached from start state by performing a BFS or DFS as follows:

DFAempty2( DFA M )

State q = q0

Stack S // Initialize a LIFO stack

Set V =// Set of visited states

S.push(q0)

while(S )

q = S.pop()

if (qF ) return NONEMPTY

V = V {q}

for (States q’ satisfying qq’ ) // an edge

if (q’ V ) S.push(q’ )

return EMPTY // Only got here if F unreachable

Q: Is this a BFS or DFS?

A: This is a DFS algorithm since stack is used. Queue would give BFS.

Acceptance problem is also solvable. There’s a rather horrible algorithm involving Chomsky Normal Form. Later we’ll see a much improved polynomial time algorithm (must have one, otherwise compilers would be useless!)

Chomsky normal form gives the following fact:

LEMMA: Suppose a grammar G is in Chomsky normal form. Let x be in L(G ), and let n be the length of x. Then x is generated by a derivation of length 2n-1, when n > 0.

LEMMA: Suppose a grammar G is in Chomsky normal form. Let x be in L(G ), and let n be the length of x. Then x is generated by a derivation of length 2n-1, when n > 0.

Proof. The first n-1 productions are of the form A BC and get us to the correct length. The last n production are unit terminating A a and derive x.

This gives rise to the following algorithm:

CFGaccept( CFG G, String x=x1 x2 x3 … xn )

CFG G’ = ChomskyNormalForm(G )

for (all derivations from start variable

of G’ of length 2n +1)

if (derivation resulted in x)

return ACCEPT

return REJECT

Q1: Why does this work for x =e?

Q2: Running time in terms of n ?

A1: Works for x =e because of “ 2n+1” clause.

A2: Exponential. If we consider the blow-up of CNF conversion, this is a truly horrendous algorithm.

- EQDFA
- ECFG
- CARDDFA