CS 221

1 / 41

# CS 221 - PowerPoint PPT Presentation

CS 221. Analysis of Algorithms Instructor: Don McLaughlin. Algorithms. What are they and Why do we care?. Algorithms. “A well defined computational procedure that takes some value or set of values, as input and produces some value or set of values as output”

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'CS 221' - buthainah-feroze

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### CS 221

Analysis of Algorithms

Instructor: Don McLaughlin

Algorithms
• What are they and
• Why do we care?
Algorithms
• “A well defined computational procedure that takes some value or set of values, as input and produces some value or set of values as output”
• “A sequence of computational steps that transforms input to the output”
• From Cormen, Leiserson, Rivest and Stein
Algorithms
• “Algorithms + Data Structures = Programs”
• Niklaus Wirth, Swiss Computer Scientist
• Or, perhaps-
• Programs = Programming_language(Algoritms + Data Structures)
Algorithms
• So you might think of an algorithm as the equation behind a function
• y=f(x)
• Just as an equation is a computational procedure for creating a certain output from a certain input
Algorithms
• “A sequence of unambiguous instructions for solving a problem, i.e. for obtaining a required output for any legitimate input in a finite amount of time”
• From Levitin, 2007
Algorithms
• So, why do we want to study them?
• Because there may be many algorithms intended to solve a problem…
• …, some are correct and some are not correct
• …and of the correct ones some are better than others (as you will see)
Algorithms
• We are largely going to be concerned about the efficiency of algorithms
• What do we mean by efficiency?
• Memory
• Time
Algorithms
• Mostly we are going to explore the efficiency of algorithms in terms of run time (T)
• In particular we are going to study the growth of algorithms in relation to the size of the problem
Algorithms
• But, how?
• Theoretically, mathematically
• Empirically
• Visually (algorithm visualization)
Theoretical Analysis of Algorithms
• To do this we need a few things-
• A computational model (what is the computer)
• A language for expressing the algorithm
• A metric and methodology for measuring the performance and efficiency of an algorithm
Theoretical Analysis of Algorithms
• RAM – Random Access Machine Model
• essentially Von Neumann architecture
• A CPU…
• …with a fixed and finite set of primitive instructions
• …which run in a fixed amount of time
• …connected to a bank of memory…
• And the CPU can read or write any memory location in one primitive operation
Theoretical Analysis of Algorithms
• RAM – Random Access Machine Model
• Primitive operations might be
• Assign a value to variable (memory location)
• Call a method or subroutine
• Arithmetic operation
• Compare two values
• Access elements of an array via index
• Return from method
• Return from algorithm
Theoretical Analysis of Algorithms
• Language for expressing an algorithm
• Programming Language
• C, VB, Java,…
• Flowcharts
Theoretical Analysis of Algorithms
• Language for expressing an algorithm
• Pseudocode
• Natural language expression of algorithm
• Really mix of natural language and language from programming language constructs
• Intended for human consumption, not computers
Theoretical Analysis of Algorithms
• Pseudo-code – has some rules
• Algorithm declaration – must name algorithm and its parameters
• Must define range of input, and domain of output
• Expressions – use <- for assignment, = for logical comparison
• Scope denoted by indentation
• Decision structures
• If – Then – Else
• While – Do loops
• Repeat – until loops
• For loops
• Method calls
• Method returns (including return from algorithm)
Theoretical Analysis of Algorithms
• Pseudo-code – an example
• arrayMax – find the largest value in an array of values

Algorithm arrayMax(A, n)

Input:An array A storing n >= 1 integers)

Output: the maximum value in the array A

currentMax <- A[0]

for i <-1 to n – 1 do

if currentMax < A[i] then

currentMax <- A[i]

return currentMax

Theoretical Analysis of Algorithms
• Insertion-Sort

Algorithm Insertion-sort(A, n)

Input:An array A storing n >= 1 integers)

Output: the array A sorted such that A0 < A1 < A2 < …An

for j <- 2 to n

do key <- A[j]

#insert A[j] into the sorted sequence

i <- j – 1

while i > 0 and A[j] > key do

A[j +1] <- A[j]

i <- i -1

A[i + 1] <- key

return A

Theoretical Analysis of Algorithms
• So now the question is - how do we determine the run-time T of these algorithms?
• Keep in mind that we need to understand the run-time of algorithms in the context of the size of the problem
• …which we will express as n
• So we are looking for T(n) … for any given algorithm
Theoretical Analysis of Algorithms
• So how are we going to do this?
• Counting
Theoretical Analysis of Algorithms
• So T(n) for arrayMax is

T(n) = 2 + 1 + n + 4(n-1) + 1

= 5n

T(n) = 2 + 1 + n + 6(n-1) + 1

= 7n-2

Theoretical Analysis of Algorithms
• So T(10) for arrayMax is
• At least 5(10) = 50
• Or
• No worse than 7(10)-2 = 68
Theoretical Analysis of Algorithms
• So, what is the run-time T(n) for the algorithm arrayMax?
• T(n)

T(n) = c1 + c2(n-1) + c3(n-1) + c4(n-1) + 1

T(n) = c1 + (n-1)(c2 + c3 +c4) + 1

Theoretical Analysis of Algorithms
• Was that every-case, best-case, worst-case?
• Why?
Theoretical Analysis of Algorithms
• So, what would T(n) be for best-case?

T(n) = c1 + c2(n-1) + c3(n-1) + 1c5

T(n) = c1 + (n-1)(c2 + c3) + 1c5

Theoretical Analysis of Algorithms
• Insertion-Sort

Algorithm Insertion-sort(A, n)

Input:An array A storing n >= 1 integers)

Output: the array A sorted such that A0 < A1 < A2 < …An

for j <- 2 to n

do key <- A[j]

#insert A[j] into the sorted sequence

i <- j – 1

while i > 0 and A[j] > key do

A[j +1] <- A[j]

i <- i -1

A[I + 1] <- key

return A

Theoretical Analysis of Algorithms
• T(n) for Insertion-Sort

T(n) = c1n + c2(n-1) + c4(n-1) + C5Σ(tj) + c6Σ(tj-1) + c7Σ(tj-1) + c8(n-1)

So… what is the best-case scenario (shortest run-time)?

Theoretical Analysis of Algorithms
• T(n) for Insertion-Sort

Ok, that would mean what in terms of the execution of the algorithm?

… that tj = 1 for all j

…therefore…

… Σ(tj) = n – 1

…and c6 and c7 drop out

Theoretical Analysis of Algorithms
• T(n) for Insertion-Sort
• So, T(n) best-case is

T(n) = c1n + c2(n-1) + c4(n-1) + c5(n-1) + c8(n-1)

= c1n + (c2 + c4 +c5 +c8)(n-1)

= (c1+c2+ c4+c5+c8)n-(c2+c4+c5+c8)

Theoretical Analysis of Algorithms
• T(n) for Insertion-Sort

T(n) = c1n + c2(n-1) + c4(n-1) + C5Σ(tj) + c6Σ(tj-1) + c7Σ(tj-1) + c8(n-1)

So… what is the worst-case scenario (longest run-time)?

Theoretical Analysis of Algorithms
• By the way - rules of thumb (for now)
• Σnj=2j = n(n+1)/2 - 1
• Σnj=2(j-1) = n(n-1)/2
Theoretical Analysis of Algorithms
• So, the worst-case

T(n) = c1n + c2(n-1) + c4(n-1) + C5(n(n+1)/2-1) + c6(n(n-1)/2) + c7(n(n-1)/2) + c8(n-1)