Algorithm

1 / 37

# Algorithm - PowerPoint PPT Presentation

Algorithm. An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time. Features of Algorithm.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Algorithm' - johana

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Algorithm
• An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.

Design and Analysis of Algorithms - Chapter 1

Features of Algorithm
• “Besides merely being a finite set of rules that gives a sequence of operations for solving a specific type of problem, an algorithm has the five important features” [Knuth1]
• finiteness (otherwise: computational method)
• termination
• definiteness
• precise definition of each step
• input (zero or more)
• output (one or more)
• effectiveness
• “Its operations must all be sufficiently basic that they can in principle be done exactly and in a finite length of time by someone using pencil and paper” [Knuth1]

Design and Analysis of Algorithms - Chapter 1

Historical Perspective
• Muhammad ibn Musa al-Khwarizmi – 9th century mathematician www.lib.virginia.edu/science/parshall/khwariz.html
• Euclid’s algorithm for finding the greatest common divisor

Design and Analysis of Algorithms - Chapter 1

Greatest Common Divisor (GCD) Algorithm 1
• Step 1 Assign the value of min{m,n} to t
• Step 2 Divide m by t. If the remainder of this division is 0, go to Step 3; otherwise, to Step 4
• Step 3 Divide n by t. If the remainder of this division is 0, return the value of t as the answer and stop; otherwise, proceed to Step 4
• Step 4 Decrease the value of t by 1. Go to Step 2
• Note: m and n are positive integers

Design and Analysis of Algorithms - Chapter 1

Correctness
• The algorithm terminates, because t is decreased by 1 each time we go through step 4, 1 divides any integer, and t eventually will reach 1 unless the algorithm stops earlier.
• The algorithm is partially correct, because when it returns t as the answer, t is the minimum value that divides both m and n

Design and Analysis of Algorithms - Chapter 1

GCD Procedure 2
• Step 1 Find the prime factors of m
• Step 2 Find the prime factors of n
• Step 3 Identify all the common factors in the two prime expansions found in Steps 1 and 2. If p is a common factor repeated i times in m and j times in n, assign to p the multiplicity min{i,j}
• Step 4 Compute the product of all the common factors with their multiplicities and return it as the GCD of m and n
• Note: as written, the algorithm requires than m and n be integers greater than 1, since 1 is not a prime

Design and Analysis of Algorithms - Chapter 1

Procedure 2 or Algorithm 2?
• Procedure 2 is not an algorithm unless we can provide an effective way to find prime factors of a number
• The sieve of Eratosthenes is an algorithm that provides such an effective procedure

Design and Analysis of Algorithms - Chapter 1

Euclid’s Algorithm
• E0 [Ensure m geq n] If m lt n, exchange m with n
• E1 [Find Remainder] Divide m by n and let r be the remainder (We will have 0 leq r lt n)
• E2 [Is it zero?] If r=0, the algorithm terminates; n is the answer
• E3 [Reduce] Set m to n, n to r, and go back to E1

Design and Analysis of Algorithms - Chapter 1

Termination of Euclid’s Algorithm
• The second number of the pair gets smaller with each iteration and cannot become negative: indeed, the new value of n is r = m mod n, which is always smaller than n. Eventually, r becomes zero, and the algorithms stops.

Design and Analysis of Algorithms - Chapter 1

Correctness of Euclid’s Algorithm
• After step E1, we have m = qn + r, for some integer q. If r = 0, then m is a multiple of n, and clearly in such a case n is the GCD of m and n. If r !=0, note that any number that divides both m and n must divide m – qn = r, and any number that divides both n and r must divide qn + r = m; so the set of divisors of {m,n} is the same as the set of divisors of {n,r}. In particular, the GCD of {m,n} is the same as the GCD of {n,r}. Therefore, E3 does not change the answer to the original problem. [Knuth1]

Design and Analysis of Algorithms - Chapter 1

Book Variation (p.4)
• Step 1 If n=0, return the value of m as the answer and stop; otherwise, proceed to Step 2.
• Step 2 Divide m by n and assign the value of the remainder to r.
• Step 3 Assign the value of n to m and the value of r to n. Go to Step 1.
• Note: m and n are nonnegative, not-both-zero integers

Design and Analysis of Algorithms - Chapter 1

Pseudocode for book version

while n <> 0 do

r <- m mod n

m <- n

n <- r

return m

Design and Analysis of Algorithms - Chapter 1

Algorithm Design Techniques
• Euclid’s algorithm is an example of a “Decrease and Conquer” algorithm: it works by replacing its instance by a simpler one (in step E3)
• It can be shown (exercise 5.6b) that the instance size, when measured by the second parameter, n, is always reduced by at least a factor of 2 after two successive iterations of Euclid’s algorithm
• In Euclid’s original, the remainder m mod n is computed by repeated subtraction of n from m
• Step E0 is not necessary for correctness; it improves time efficiency

Design and Analysis of Algorithms - Chapter 1

Notion of algorithm

problem

algorithm

“computer”

input

output

Algorithmic solution

Design and Analysis of Algorithms - Chapter 1

Example of computational problem: sorting
• Statement of problem:
• Input: A sequence of n numbers <a1, a2, …, an>
• Output: A reordering of the input sequence <a´1, a´2, …, a´n> so that a´i≤ a´j whenever i < j
• Instance: The sequence <5, 3, 2, 8, 3>
• Algorithms:
• Selection sort
• Insertion sort
• Merge sort
• (many others)

Design and Analysis of Algorithms - Chapter 1

Selection Sort
• Input: array a[1],…,a[n]
• Output: array a sorted in non-decreasing order
• Algorithm:
• for i=1 to n

swap a[i] with smallest of a[i+1],…a[n]

Design and Analysis of Algorithms - Chapter 1

Sorting: Some Terms
• Key
• Stable sorting
• In place sorting

Design and Analysis of Algorithms - Chapter 1

Graphs: Some Terms
• Graph
• Vertices, edges
• Undirected, directed
• Loops
• Complete, dense, sparse
• Paths: sequences of vertices or of edges?
• Path length: number of edges in path
• Walk: repeated vertices and edges allowed
• Path: repeated vertices are allowed, repeated edges are not allowed
• Simple path: neither repeated vertices nor repeated edges are allowed
• Cycle: simple path that starts and ends at the same vertex

Design and Analysis of Algorithms - Chapter 1

Some Well-known Computational Problems
• Sorting
• Searching
• Shortest paths in a graph
• Minimum spanning tree
• Primality testing
• Traveling salesman problem
• Knapsack problem
• Chess
• Towers of Hanoi
• Program termination

Design and Analysis of Algorithms - Chapter 1

Basic Issues Related to Algorithms
• How to design algorithms
• How to express algorithms
• Proving correctness
• Efficiency
• Theoretical analysis
• Empirical analysis
• Optimality

Design and Analysis of Algorithms - Chapter 1

Brute force

Divide and conquer

Decrease and conquer

Transform and conquer

Algorithm design strategies

See http://www.aw-bc.com/info/levitin/taxonomy.html

This is not a unique taxonomy.

• Greedy approach
• Dynamic programming
• Backtracking and Branch and bound
• Space and time tradeoffs

Design and Analysis of Algorithms - Chapter 1

Analysis of Algorithms
• How good is the algorithm?
• Correctness
• Time efficiency
• Space efficiency
• Does there exist a better algorithm?
• Lower bounds
• Optimality

Design and Analysis of Algorithms - Chapter 1

What is an algorithm?
• Recipe, process, method, technique, procedure, routine,… with the following requirements:
• Finiteness
• terminates after a finite number of steps
• Definiteness
• rigorously and unambiguously specified
• Input
• valid inputs are clearly specified
• Output
• can be proved to produce the correct output given a valid input
• Effectiveness
• steps are sufficiently simple and basic

Design and Analysis of Algorithms - Chapter 1

Why study algorithms?
• Theoretical importance
• the core of computer science
• Practical importance
• A practitioner’s toolkit of known algorithms
• Framework for designing and analyzing algorithms for new problems

Design and Analysis of Algorithms - Chapter 1

Correctness
• Termination
• Well-founded sets: find a quantity that is never negative and that always decreases as the algorithm is executed
• Partial Correctness
• For recursive algorithms: induction
• For iterative algorithms: axiomatic semantics, loop invariants

Design and Analysis of Algorithms - Chapter 1

Proving Correctness
• Make sure that the algorithm will terminate.
• Define a precondition (allowable input and state) and a postcondition (correct output).
• Show that, given the precondition, algorithm will fulfill the postcondition.
• In many cases, the algorithm is either recursive or iterative. Determine which one applies.

Design and Analysis of Algorithms - Chapter 1

Recursive Algorithms

Use induction and show correct fulfillment of postcondition in:

• basis case (input is of size 0)
• case where input is of size n+1, assuming (inductive assumption) that postcondition is fulfilled when input is of size n (simple induction) or input is of all sizes from 0 to n (strong or complete induction)

Design and Analysis of Algorithms - Chapter 1

Iterative Algorithms
• Determine an appropriate invariant.
• Show that the invariant is true before entering the loop for the first time.
• Show that, if the invariant is true and the loop condition is false, the postcondition is fulfilled.
• Show, by induction, that, if the invariant is true for input of size n and the loop condition is true, the invariant is also true for input of size n+1.

Design and Analysis of Algorithms - Chapter 1

Example: Recursive Factorial

(* Pre: i GEQ 0 *)

fun fact(i) = if i = 0 then 1 else i * fact(i-1);

(* Post: fact(i) = i! *)

Design and Analysis of Algorithms - Chapter 1

Example: Iterative Factorial

(* Pre’: i>= 0*)

fun fact(i)

int j=1, f=1;

(* Pre: i>=0 and j=f=1 *)

while j <= i

f = f * j;

j := j+1;

(* Post: f = i! *)

return f;

(* Post’: fact(i) = i! *)

Design and Analysis of Algorithms - Chapter 1

Example: Product Algorithm
• To be added

Design and Analysis of Algorithms - Chapter 1

Example: Recursive BinarySearch
• To be added

Design and Analysis of Algorithms - Chapter 1

Example: Iterative Binary Search
• To be added

Design and Analysis of Algorithms - Chapter 1

Example: Recursively Computing Binomial Coefficients
• Multiple calls
• To be added

Design and Analysis of Algorithms - Chapter 1

Example: RecursiveSelectionSort

(* Author: Kingston *)(* Pre: a LTE b + 1 *)(* SelectionSort(A, a, b) *)(* Post: A[a] LTE A[a+1] LTE A[a+2] LTE ... A[b] *)(* and A is a permutation of the input array *)procedure SelectionSort (var A: AType; a,b: integer);var i: integer;begin if a = b + 1 then (* do nothing *) else i := MinIndex(A, a, b); if (i <> a) then Swap(A[i],A[a]); end; SelectionSort(A, a + 1, b); end;end SelectionSort;(* MinIndex returns the index of a minimum element of the non-empty arrayA[i..j], and Swap(A[i],A[j] swaps A[i] and A[j] *)

Design and Analysis of Algorithms - Chapter 1

Example: Max Contiguous Subvector
• Algorithm:Maximum-Sum Contiguous Subvector
• /* Precondition: x[1..n] is an array of floating-point (real) numbers. */
• procedureMaxSumContigVector (x[1..n]: float)// Returns the maximum sum of a continuous subarray drawn from x.MaxSoFar := 0.0MaxEndingHere := 0i := 1while i ≠ n + 1 doMaxEndingHere := max(0.0, MaxEndingHere + x[i])MaxSoFar := max(MaxSoFar, MaxEndingHere)i := i + 1
• endreturn MaxSoFar
• /* Postcondition: MaxSoFar is the maximum sum of any contiguous subvector in x[1..n]. */

Design and Analysis of Algorithms - Chapter 1

Complexity
• Space complexity
• Time complexity
• For iterative algorithms: sums
• For recursive algorithms: recurrence relations

Design and Analysis of Algorithms - Chapter 1