1 / 41

Exploring Algorithms: Efficiency Study & Theoretical Analysis

Dive into the world of algorithms - from computational procedures to programming languages. Understand their efficiency through theoretical analysis in the context of problem size. RAM Model, Pseudo-code, and runtime calculations are key elements.

buthainah-feroze
Download Presentation

Exploring Algorithms: Efficiency Study & Theoretical Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS 221 Analysis of Algorithms Instructor: Don McLaughlin

  2. Algorithms • What are they and • Why do we care?

  3. Algorithms • “A well defined computational procedure that takes some value or set of values, as input and produces some value or set of values as output” • “A sequence of computational steps that transforms input to the output” • From Cormen, Leiserson, Rivest and Stein

  4. Algorithms • “Algorithms + Data Structures = Programs” • Niklaus Wirth, Swiss Computer Scientist • Or, perhaps- • Programs = Programming_language(Algoritms + Data Structures)

  5. Algorithms • So you might think of an algorithm as the equation behind a function • y=f(x) • Just as an equation is a computational procedure for creating a certain output from a certain input

  6. Algorithms • “A sequence of unambiguous instructions for solving a problem, i.e. for obtaining a required output for any legitimate input in a finite amount of time” • From Levitin, 2007

  7. Algorithms • So, why do we want to study them? • Because there may be many algorithms intended to solve a problem… • …, some are correct and some are not correct • …and of the correct ones some are better than others (as you will see)

  8. Algorithms • We are largely going to be concerned about the efficiency of algorithms • What do we mean by efficiency? • Memory • Time

  9. Algorithms • Mostly we are going to explore the efficiency of algorithms in terms of run time (T) • In particular we are going to study the growth of algorithms in relation to the size of the problem

  10. Algorithms • But, how? • Theoretically, mathematically • Empirically • Visually (algorithm visualization)

  11. Theoretical Analysis of Algorithms • To do this we need a few things- • A computational model (what is the computer) • A language for expressing the algorithm • A metric and methodology for measuring the performance and efficiency of an algorithm

  12. Theoretical Analysis of Algorithms • RAM – Random Access Machine Model • essentially Von Neumann architecture • A CPU… • …with a fixed and finite set of primitive instructions • …which run in a fixed amount of time • …connected to a bank of memory… • And the CPU can read or write any memory location in one primitive operation

  13. Theoretical Analysis of Algorithms • RAM – Random Access Machine Model • Primitive operations might be • Assign a value to variable (memory location) • Call a method or subroutine • Arithmetic operation • Compare two values • Access elements of an array via index • Return from method • Return from algorithm

  14. Theoretical Analysis of Algorithms • Language for expressing an algorithm • Programming Language • C, VB, Java,… • Flowcharts

  15. Theoretical Analysis of Algorithms • Language for expressing an algorithm • Pseudocode • Natural language expression of algorithm • Really mix of natural language and language from programming language constructs • Intended for human consumption, not computers

  16. Theoretical Analysis of Algorithms • Pseudo-code – has some rules • Algorithm declaration – must name algorithm and its parameters • Must define range of input, and domain of output • Expressions – use <- for assignment, = for logical comparison • Scope denoted by indentation • Decision structures • If – Then – Else • While – Do loops • Repeat – until loops • For loops • Method calls • Method returns (including return from algorithm)

  17. Theoretical Analysis of Algorithms • Pseudo-code – an example • arrayMax – find the largest value in an array of values Algorithm arrayMax(A, n) Input:An array A storing n >= 1 integers) Output: the maximum value in the array A currentMax <- A[0] for i <-1 to n – 1 do if currentMax < A[i] then currentMax <- A[i] return currentMax

  18. Theoretical Analysis of Algorithms • Insertion-Sort Algorithm Insertion-sort(A, n) Input:An array A storing n >= 1 integers) Output: the array A sorted such that A0 < A1 < A2 < …An for j <- 2 to n do key <- A[j] #insert A[j] into the sorted sequence i <- j – 1 while i > 0 and A[j] > key do A[j +1] <- A[j] i <- i -1 A[i + 1] <- key return A

  19. Theoretical Analysis of Algorithms • So now the question is - how do we determine the run-time T of these algorithms? • Keep in mind that we need to understand the run-time of algorithms in the context of the size of the problem • …which we will express as n • So we are looking for T(n) … for any given algorithm

  20. Theoretical Analysis of Algorithms • So how are we going to do this? • Counting

  21. Theoretical Analysis of Algorithms

  22. Theoretical Analysis of Algorithms

  23. Theoretical Analysis of Algorithms • So T(n) for arrayMax is T(n) = 2 + 1 + n + 4(n-1) + 1 = 5n T(n) = 2 + 1 + n + 6(n-1) + 1 = 7n-2

  24. Theoretical Analysis of Algorithms • So T(10) for arrayMax is • At least 5(10) = 50 • Or • No worse than 7(10)-2 = 68

  25. Theoretical Analysis of Algorithms

  26. Theoretical Analysis of Algorithms

  27. Theoretical Analysis of Algorithms

  28. Theoretical Analysis of Algorithms • So, what is the run-time T(n) for the algorithm arrayMax? • T(n) T(n) = c1 + c2(n-1) + c3(n-1) + c4(n-1) + 1 T(n) = c1 + (n-1)(c2 + c3 +c4) + 1

  29. Theoretical Analysis of Algorithms • Was that every-case, best-case, worst-case? • Why?

  30. Theoretical Analysis of Algorithms • So, what would T(n) be for best-case? T(n) = c1 + c2(n-1) + c3(n-1) + 1c5 T(n) = c1 + (n-1)(c2 + c3) + 1c5

  31. Theoretical Analysis of Algorithms • Insertion-Sort Algorithm Insertion-sort(A, n) Input:An array A storing n >= 1 integers) Output: the array A sorted such that A0 < A1 < A2 < …An for j <- 2 to n do key <- A[j] #insert A[j] into the sorted sequence i <- j – 1 while i > 0 and A[j] > key do A[j +1] <- A[j] i <- i -1 A[I + 1] <- key return A

  32. Theoretical Analysis of Algorithms

  33. Theoretical Analysis of Algorithms

  34. Theoretical Analysis of Algorithms • T(n) for Insertion-Sort T(n) = c1n + c2(n-1) + c4(n-1) + C5Σ(tj) + c6Σ(tj-1) + c7Σ(tj-1) + c8(n-1) So… what is the best-case scenario (shortest run-time)?

  35. Theoretical Analysis of Algorithms • T(n) for Insertion-Sort Ok, that would mean what in terms of the execution of the algorithm? … that tj = 1 for all j …therefore… … Σ(tj) = n – 1 …and c6 and c7 drop out

  36. Theoretical Analysis of Algorithms • T(n) for Insertion-Sort • So, T(n) best-case is T(n) = c1n + c2(n-1) + c4(n-1) + c5(n-1) + c8(n-1) = c1n + (c2 + c4 +c5 +c8)(n-1) = (c1+c2+ c4+c5+c8)n-(c2+c4+c5+c8)

  37. Theoretical Analysis of Algorithms

  38. Theoretical Analysis of Algorithms • T(n) for Insertion-Sort T(n) = c1n + c2(n-1) + c4(n-1) + C5Σ(tj) + c6Σ(tj-1) + c7Σ(tj-1) + c8(n-1) So… what is the worst-case scenario (longest run-time)?

  39. Theoretical Analysis of Algorithms • By the way - rules of thumb (for now) • Σnj=2j = n(n+1)/2 - 1 • Σnj=2(j-1) = n(n-1)/2

  40. Theoretical Analysis of Algorithms • So, the worst-case T(n) = c1n + c2(n-1) + c4(n-1) + C5(n(n+1)/2-1) + c6(n(n-1)/2) + c7(n(n-1)/2) + c8(n-1)

More Related