1 / 20

# Overview of the theory of computation - PowerPoint PPT Presentation

Overview of the theory of computation. Episode 3. Turing machines The traditional concepts of computability, decidability and recursive enumerability The limitations of the power of Turing machines The Church-Turing thesis Mapping reducibilty Turing reducibility

Related searches for Overview of the theory of computation

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Overview of the theory of computation' - christmas

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Episode 3

• Turing machines

• The traditional concepts of computability, decidability and recursive enumerability

• The limitations of the power of Turing machines

• The Church-Turing thesis

• Mapping reducibilty

• Turing reducibility

• Kolmogorov complexity

0

Components of a Turing machine (TM)

Input tape

2 0 - - - - - - -

Work tape

4 0 0 - 1 # 2 \$ - -

Output tape

4 0 0 - - - - - -

No direct access

Control

(transition function)

- blank symbol

Has a finite number of states, two of which, Start and Halt, are special.

How a TM works

Input tape

2 0 - - - - - - -

Work tape

- - - - - - - - - -

Output tape

- - - - - - - - -

• At the beginning, the machine is in its Start state, the input tape has an input on it,

• the other tapes are blank, and the two scanning heads are in their leftmost positions.

• Such a situation is shown above, with input being20.

• After the computation starts, the machine goes from one state to another, writes on the

• work tape, and moves its scanning heads left or right according to its transition function,

• which precisely prescribes what to do in each particular case, depending on the current

• state and the contents of the two scanned cells (on the input and work tapes).

• If and when the machine enters the Halt state, ithalts, and whatever string w is to

• the left of the work-tape scanning head, will be automatically copied onto the output

• tape. Such a string w is considered to be the output of the machine. Slide 3.1 shows

• such a situation, with output 400.

Variants of Turing machines

• There are numerous reasonable variations of the Turing machine

• model, and they all turn out to be equivalent. For example:

• More often than not, there is just a single tape for everything (input,

• output, work).

• Some models allow multiple work tapes.

• The tape can be infinite in both directions (have no beginning).

• And so on.

One of the advantages of the particular model we have chosen here is

that it can be easily adapted to cases where infinitely many inputs and

outputs are allowed. In such cases, entering the Halt state will only have

the effect of sending a (next) output to the output tape, but otherwise the

machine would not halt.

2 0 - 3 - 1 0 - 2 - ...

Input

Output

4 0 0-9- 10 0 - ...

Computability

Definition 3.1. Let f: ABbe a function.

(a) We say that a TMMcomputes fiff, for every stringwA,

vsuch thatv = f(w).

(b) When such a machineMexists, we say thatfis computable.

There is no need to separately consider functions with more than one

(but finitely many) arguments, as tuples of strings can be encoded

through (just) strings.

For example, the function f(x,y)=x+y can be thought of as a

single-argument function that returns string “3” on input string “(2,1)”,

returns string “20” on input string “(8,12)”, etc.

Decidability

Definition 3.2. Let Sbe a set of strings over a given alphabetA.

(a) We say that a TMMdecides Siff, for every stringwA*,

1 (or “yes”, or “accept”) if wS;

0 (or “no”, or “reject”)ifwS.

(b) When such a machineMexists, we say thatSis decidable.

As every set S can be thought of as and identified with its characteristic

function, decidability is just a special case of computability.

The concept of decidability extends to predicates and relations,

as the latter can be thought of as sets.

Recursive enumerability

Definition 3.3. Let Sbe a set of strings over a given alphabetA.

(a) We say that a TMMrecognizes Siff, for every stringwA*,

ifwS, thenM(halts and) outputs“yes”;

ifwS, then M either (halts and) outputs something else

(such as “no”), or never halts.

(b) When such a machineMexists, we say thatSis recursively

enumerable.

Again, this definition extends from sets to predicates and relations.

Computational problems in the traditional sense

Computational problems in the traditional sense, as established in

the traditional theory of computation, are nothing but functions,

sets, predicates or relations (to be computed or decided or recognized).

“function”, “relation”, etc.

For example, we may refer to the function f(x)=x2 as “the problem of

finding the square of a given number”, and refer to the set (predicate,

relation) {x | x is an even number} as “the problem of telling whether

a given number is even”, or “the evennees problem”, etc.

The limitations of the power of TMs

Alas, not all reasonable problems can be handled by TMs.

Examples of undecidable or incomputable problems:

1. The acceptance problem for Turing machines, i.e. the problem of telling whether a

given Turing machine accepts (meaning outputting “yes”) a given input.

2. The halting problem for Turing machines, i.e. the problem of telling whether a

given Turing machine ever halts on a given input.

3. The problem of finding the Kolmogorov complexity of a given number.

4. The problem of telling whether a given polynomial equation has integral roots.

5. The problem of first-order logical validity.

Examples of problems that are not recursively enumerable:

6. The complements of the above problems.

7. The problem of second-order logical validity.

8. The problem of telling whether a given formula of arithmetic is true.

=

Computers

intuitive concept

mathematical concept

intuitive concept

The Church-Turing thesis

=

Turing machines

Algorithms

Of course, here “=” should be understood not in the strict sense, but

in the sense of equivalence of power.

That is, according to the Church-Turing thesis, the problems that have

algorithmic solutions also have Turing machine solutions (deciders,

recognizers, etc.), and vice versa.

Thus, the existence of undecidable or incomputable problems means

the existence of problems that have no algorithmic solutions, problems

that no computers can ever handle.

• Definition 3.4. Let A and B be sets of strings over an alphabet .

• (a) We say that a function f: ** is a mapping reduction

• from A to B iff f is computable and, for every w*,

• wAifff(w)B.

• (b) When such a function f exists, we say that A is mapping

• reducible to B, and write AmB. In the literature, mapping

• reducibility is more often called “many-one reducibility”.

Definition of mapping reducibility

*

*

A

B

f

f

• Definition 3.4. Let A and B be sets of strings over an alphabet .

• (a) We say that a function f: ** is a mapping reduction

• from A to B iff f is computable and, for every w*,

• wAifff(w)B.

• (b) When such a function f exists, we say that A is mapping

• reducible to B, and write AmB. In the literature, mapping

• reducibility is more often called “many-one reducibility”.

Definition of mapping reducibility

*

*

A

B

f

f

• Definition 3.4. Let A and B be sets of strings over an alphabet .

• (a) We say that a function f: ** is a mapping reduction

• from A to B iff f is computable and, for every w*,

• wAifff(w)B.

• (b) When such a function f exists, we say that A is mapping

• reducible to B, and write AmB. In the literature, mapping

• reducibility is more often called “many-one reducibility”.

Definition of mapping reducibility

*

*

A

B

f

f

Using mapping reducibility for proving decidability/undecidability

Theorem 3.5. If AmBandBis decidable, thenAis decidable.

Proof: Let DB be a decider for B and f be a mapping reduction from

A to B. We describe a decider DA for A as follows.

DA= “On input w:

1. Compute f(w).

2. Run DB on input f(w) and output whatever DB outputs.” 

( means “end of proof ”).

Thus, the decidability of a problem A can be proven by finding a mapping reduction

from A to some problem B which is already known to be decidable.

Or, the undecidability of a problem B can be proven by finding a mapping reduction

from A to B, where A is a problem already known to be undecidable.

Theorem 3.5 remains valid with “recursively enumerable” instead

of “decidable”.

For every TM M, let M’ be the following TM:

M’ = “On input x:

1. Run M on x.

2. If M outputs “yes”, accept.

3. If M outputs anything else, enter an infinite loop.”

A mapping reduction of the acceptance problem to the halting problem

Thus,

• If M outputs “yes” on input x, then M’

• If M outputs anything else on input x, then M’

• If M never halts on input x, then M’

• To summarize, M accepts x iff M’

accepts x

never halts on x

never halts on x

halts on x

Let then f be the function defined by f(M,w)=(M’,w).

Is f computable?

Of course.

And, since we have

(M,w)ACCEPTANCE_PROBLEM iff f(M,w)HALTING_PROBLEM,

f is a mapping reduction of the acceptance problem to the halting

problem.

Definition of Turing reducibility

An oracle for a set (relation) B is an external device that is capable of

reporting whether any given string w is a member of B.

An oracle Turing machine (OTM) is a modified Turing machine that

has the additional capability of querying an oracle.

Example: Construct an OTM O with an oracle for the acceptance

problem, such that O decides thenonacceptance problem(the

complement of the acceptance problem).

O = “On input (M,w), where M is a TM and w is a string:

1. Query the oracle to determine whether Macceptsw.

2. If the oracle answers NO, accept; if YES, reject.”

Definition 3.6. We say that a problem A is Turing reducibleto a

problem B, written ATB, iff there is an OTM M with an oracle for

B, such that M decides A (or M computes A, if A is a function

rather than a relation).

Using Turing reducibility for proving decidability/undecidability

Theorem 3.7. If ATB and B is decidable, then A is decidable.

Proof. If B is decidable, then we may replace the oracle for B by an

actual procedure that decides B. Thus we may replace the OTM that

(using an oracle for B) decides A by an ordinary TM that decides A. 

Does this proof go through for “recursively enumerable” instead of

“decidable”? (see the end of Slide 3.11)

No! In fact, one can prove the opposite. For example, as we saw on the previous

slide, the nonacceptance problem is Turing reducible to the acceptance problem

(generally, any problem is Turing reducible to its complement). And the acceptance

problem is obviously recursively enumerable (why?). But the nonacceptance

problem is not recursively enumerable (why?).

It also follows from the comment at the end of Slide 3.11 that the nonacceptance

problem is not mapping reducible to the acceptance problem.

Yes!

Does mapping reducibility always imply Turing reducibility?

Turing reducibility is the weakest form of reducibility.

Turing reductions of the acceptance problem to the halting problem

Example: Show that the acceptance problem is Turing reducible to the halting problem.

Solution 1. When receiving the question “Does M accept w?”, replace M by M’ (as

described on Slide 3.12), and ask the oracle whether M’ halts on w. Repeat whatever

answer you get from the oracle. 

Solution 2. When receiving the question “Does M accept w?”, ask the oracle if M halts

on input w.

If the oracle says “No”, you also say “No”.

If the oracle says “Yes”, start running (simulating) machine M on input w until

you see it has halted. If its output is “Yes”, you also say “Yes”. Otherwise say “No”. 

Why could not we just do simulation as described in the last paragraph of Solution 2,

without any preliminary use of the oracle?

Because there would be no guarantee that

the simulation would ever end.

All Turing machines can be listed in the lexicographic order of their descriptions:

M0, M1, M2, M3, ..., Mi, ....

Number i can thus be considered the code of the machine Mi.

Kolmogorov complexity

Definition 3.8. Let m be a natural number. The Kolmogorov

complexity of m is the smallest number i such that machine Mi

outputs m on input 0.

Note: More often, Kolmogorov complexity is defined not as the

above number i itself, but as the size|i| of that number, i.e. the logarithm

of i. For our purposes, however, this makes no difference, and we will

stick to Definition 3.8 as it is given.

Importance: Kolmogorov complexity can be seen as a mathematical counterpart of the intuitive concepts of “randomness” or “amount of information”. The greater the Kolmogorov complexity of a given object,

Turing reduction of the Kolmogorov complexity problem to the halting problem

Example: Show that the Kolmogorov complexity problem (i.e. the problem of finding

the Kolmogorov complexity of a given number) is Turing reducible to the halting

problem.

Solution. After receiving the question “What is the Kolmogorov complexity of m?”,

initialize variable i to 0, and do the following:

1. Ask the oracle if the machine Mihalts on input0.

2. If the oracle says “No”, increment i to i+1, and go back to Step 1.

3. If the oracle says “Yes”, simulate Mi on input 0 until it halts. If you see that the

output of Mi is m, return i as your output. Otherwise, increment i to i+1 and go back to

Step 1. 

Unlike the example on

Slide 3.15, here the oracle is queried more than once. The number of queries, however,

can be shown to be bounded by a certain linear function of m.

How many times does the above algorithm use the oracle?

Theorem 3.9. The Kolmogorov complexity problem is not computable.