1 / 56

Algorithm Analysis

Algorithm Analysis. 1. Purpose. Why bother analyzing code; isn’t getting it to work enough? Estimate time and memory in the average case and worst case Identify bottlenecks, i.e., where to reduce time Compare different approaches Speed up critical algorithms.

yuli-hays
Download Presentation

Algorithm Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Algorithm Analysis 1

  2. Purpose • Why bother analyzing code; isn’t getting it to work enough? • Estimate time and memory in the average case and worst case • Identify bottlenecks, i.e., where to reduce time • Compare different approaches • Speed up critical algorithms

  3. Problem – Algorithm – Data Structures & Techniques Many algorithms can exist to solve thesame problem Algorithm (e.g., binary search) Problem (e.g., searching) solves contains uses Data structures & Techniques (e.g., sorted array) Specification (Input => Output) When designing algorithms,we may end up breakingthe problem into smaller“sub-problems”

  4. Algorithm • Algorithm • Well-defined computational procedure for transforming inputs to outputs • Problem • Specifies the desired input-output relationship • Correct algorithm • Produces the correct output for every possible input in finite time • Solves the problem

  5. Algorithm Analysis • Predict resource utilization of an algorithm • Running time • Memory • Dependent on architecture • Serial • Parallel • Quantum

  6. Factors for Algorithmic Design Consideration • Run-time • Space/memory • Suitability to the problem’s application domain (contextual relevance) • Scalability • Guaranteeing correctness • Deterministic vs. randomized • Computational model • System considerations: cache, disk, network speeds, etc.

  7. Model that we will assume • Simple serial computing model CPU RAM (mainmemory) control Secondary storage(disk) I/O ALU INPUT Keyboard, mouse devices OUTPUT Monitor, printer registers

  8. Other Models • Multi-core models • Co-processor model • Multi-processor model • Shared memory machine • Multiple processors sharing common RAM • Memory is cheap ($20 per GB) • Cray XMT Supercomputer • Up to 64 TB (65,536 GB) shared memory • Up to 8000 processors • 128 independent threads per processor • $150M

  9. Other Models Distributed memory model www.top500.org Supercomputer speed is measured in number offloating point operations per sec (or FLOPS) • Fastest supercomputer (as of June 2010): • Jaguar @ Oak Ridge National Lab • 2.3 PetaFlop/s (or 2.3 x 1015) (floating points operations per sec) • Cray XT5 (w/ AMD Opteron 6-core 2.6GHz) • #processing cores: 224,256 • #aggregate memory: 300 TB • Price tag: millions of $$$ 9

  10. What to Analyze • Running time T(N) • N (or n) is typically the size of the input • Sorting? • Multiplying two integers? • Multiplying two matrices? • Traversing a graph? • T(N) measures number of primitive operations performed • E.g., addition, multiplication, comparison, assignment

  11. Example for calculating T(n) #operations int sum (int n) 0 { int partialSum; 0 partialSum = 0; 1 for (int i = 1; i <= n; i++) 1+(n+1)+n partialSum += i * i * i; n*(1+1+2) return partialSum; 1 } T(n) = 6n+4

  12. Example: Another less-precise but equally informative analysis #operations int sum (int n) 0 { int partialSum; 0 partialSum = 0; 1 for (int i = 1; i <= n; i++) n partialSum += i * i * i; n*1 return partialSum; 1 } T(n) n 12

  13. Do constants matter? • What happens if: • N is small (< 100)? • N is medium (<1,000,000)? • N is large (> 1,000,000)? • Asymptotically, curves matter more than absolute values! • Let: • T1(N) = 3N2 + 100N + 1 • T2(N) = 50N2 + N + 500 • Compare T1(N) vs T2(N)?

  14. Big-O O() Omega Ω() small-O o() small-omega () Theta () Worst-case Average-case Best-case “Algorithms” Lower-bound Upper-bound Tight-bound “Optimality” Algorithmic Notation & Analysis Describes the input Describes the problem & its solution Asymptotic notations that help us quantify & compare costsof different solutions

  15. Asymptotic Notation • Theta • Tight bound • Big-oh • Upper bound • Omega • Lower bound The main idea: Express cost relative to a standard function curve (e.g., lg n, n, n2, etc.)

  16. Some Standard Function Curves cubic quadratic linear (curves not to scale) Initial aberrations do NOT matter! 16

  17. Big-oh : O() What you want to measure A standard function (e.g., n, n2) ≤ • f(N) = O(g(N)) if there exist positive constants c and n0 such that: • f(N) ≤ c*g(N) when N>n0 • Asymptotic upper bound, possibly tight Upper bound for f(N)

  18. c g(n) = n2 f(n) = 10 n n0 Up & down fluctuation allowed in this zone Example for big-Oh • E.g., let f(n)=10 n • ==> f(n) = O(n2) • Proof: • If f(n) = O(g(n)) • ==> f(n) ≤ c g(n), for all n>n0 • (show such a <c,n0> combination exists) • ==> c=1, n0 = 10 • (Remember: always try to find the lowest possible n0) c=1 Breakevenpoint (n0)

  19. Ponder this • If f(n) = 10 n, then: • Is f(n) = O(n)? • Is f(n) = O(n2)? • Is f(n) = O(n3)? • Is f(n) = O(n4)? • Is f(n) = O(2n)? • … • If all of the above, then what is the best answer? • f(n) = O(n) Yes: for <c=10, n0=1> Yes: for <c=1, n0=1>

  20. Little-oh : o() < • f(N) = o(g(N)) if there exist positive constants c and n0 such that: • f(N)< c*g(N) when N>n0 • E.g., f(n)=10 n; g(n) = n2 • ==> f(n) = o(g(n)) Strict upper bound for f(N)

  21. Big-omega : Ω () ≥ • f(N) = Ω(g(N)) if there exist positive constants c and n0 such that: • f(N) ≥ c*g(N) when N>n0 • E.g., f(n)= 2n2; g(n) =n log n ==> f(n) = Ω (g(n)) Lower bound for f(N), possibly tight

  22. Little-omega : () > • f(N) = (g(N)) if there exist positive constants c and n0 such that: • f(N) > c*g(N) when N>n0 • E.g., f(n)= 100 n2; g(n) =n ==> f(n) =  (g(n)) Strict lower bound for f(N)

  23. Theta : Θ() f(N) = Θ(h(N)) if there exist positive constants c1,c2 and n0 such that: c1h(N) ≤f(N) ≤ c2h(N) for all N>n0 (≡): f(N) = Θ(h(N)) if and only if f(N) = O(h(N)) and f(N) = Ω(h(N)) ≡ Tight bound for f(N) 23

  24. Example (for theta) • f(n) = n2 – 2n  f(n) = Θ(?) • Guess:f(n) = Θ(n2) • Verification: • Can we find valid c1, c2, and n0? • If true:  c1n2≤f(n) ≤ c2n2  …

  25. The Asymptotic Notations

  26. Asymptotic Growths c2 g(n) c g(n) f(n) f(n) f(n) c1 g(n) c g(n) n n n n0 n0 n0 f(n) = ( g(n) ) f(n) = O( g(n) )f(n) = o( g(n) ) f(n) = ( g(n) ) f(n) =  ( g(n) )

  27. Rules of Thumb while using Asymptotic Notations Algorithm’s complexity: • When asked to analyze an algorithm’s complexities: 1st Preference: Whenever possible, use () 2nd Preference: If not, use O() - or o() 3rd Preference: If not, use () - or () Tight bound Upper bound Lower bound

  28. Rules of Thumb while using Asymptotic Notations… Algorithm’s complexity: • Always express an algorithm’s complexity in terms of its worst-case, unless otherwise specified

  29. Rules of Thumb while using Asymptotic Notations… Problem’s complexity • Ways to answer a problem’s complexity: Q1) This problem is at least as hard as … ? Use lower bound here Q2) This problem cannot be harder than … ? Use upper bound here Q3) This problem is as hard as … ? Use tight bound here

  30. A few examples • N2 = O(N2) = O(N3) = O(2N) • N2 = Ω(1) = Ω(N) = Ω(N2) • N2 = Θ(N2) • N2 = o(N3) • 2N2 + 1 = Θ(?) • N2 + N = Θ(?) • N3 - N2 = Θ(?) • 3N3 – N3 = Θ(?)

  31. Reflexivity • Is f(n) = Θ(f(n))? • Is f(n) = O(f(n))? • Is f(n) = Ω(f(n))? • Is f(n) = o(f(n))? • Is f(n) = (f(n))? yes reflexive yes yes no Not reflexive no

  32. Symmetry • f(n) = Θ(g(n)) “iff” g(n) = Θ(f(n)) “If and only if”

  33. Transpose symmetry • f(n) = O(g(n)) iff g(n) = Ω(f(n)) • f(n) = o(g(n)) iff g(n) = w(f(n))

  34. Transitivity • f(n) = Θ(g(n)) andg(n) = Θ(h(n)) f(n) = Θ(h(n)) • f(n) = O(g(n)) andg(n) = O(h(n)) ? • f(n) = Ω(g(n)) andg(n) = Ω(h(n)) ? • …

  35. additive multiplicative More rules… • Rule 1: If T1(n) = O(f(n)) and T2(n) = O(g(n)), then • T1(n) + T2(n) = O(f(n) + g(n)) • T1(n) * T2(n) = O(f(n) * g(n)) • Rule 2: If T(n) is a polynomial of degree k, then T(n) = Θ(nk) • Rule 3: logk n = O(n) for any constant k • Rule 4: logan = (logbn) for any constants a & b

  36. Some More Proofs • Prove that: n lg n = O(n2) • We know lg n ≤ n for n≥1 ( n0=1) • Multiplying n on both sides: n lg n ≤ n2 n lg n ≤ 1. n2

  37. Some More Proofs… • Prove that: 6n3≠O(n2) • By contradiction: • If 6n3=O(n2) • 6n3≤c n2 • 6n ≤c • It is not possible to bound a variable with a constant • Contradiction

  38. Maximum subsequence sum problem • Given N integers A1, A2, …, AN, find the maximum value (≥0) of: • Don’t need the actual sequence (i,j), just the sum • If final sum is negative, output 0 • E.g., [1, -4, 4, 2, -3, 5, 8, -2] 16 is the answer j i

  39. MaxSubSum: Solution 1 • Compute for all possible subsequence ranges (i,j) and pick the maximum range MaxSubSum1 (A) maxSum = 0 for i = 1 to N for j = i to N sum = 0 for k = i to j sum = sum + A[k] if (sum > maxSum) then maxSum = sum return maxSum (N) All possible start points (N) All possible end points (N) Calculate sum for range [ i .. j ] Total run-time = (N3)

  40. Solution 1: Analysis • More precisely = (N3)

  41. New code: MaxSubSum: Solution 2 New sum A[j] Old sum • Observation: • ==> So, re-use sum from previous range Old code: MaxSubSum2 (A) maxSum = 0 for i = 1 to N sum = 0 for j = i to N sum = sum + A[j] if (sum > maxSum) then maxSum = sum return maxSum MaxSubSum1 (A) maxSum = 0 for i = 1 to N for j = i to N sum = 0 for k = i to j sum = sum + A[k] if (sum > maxSum) then maxSum = sum return maxSum (N) (N) Total run-time = (N2) Sum (new k) = sum (old k) + A[k] => So NO need to recompute sum for range A[i..k-1]

  42. Solution 2: Analysis Can we do better than this? Use a Divide & Conquertechnique?

  43. MaxSubSum: Solution 3 • Recursive, “divide and conquer” • Divide array in half • A1..center and A(center+1)..N • Recursively compute MaxSubSum of left half • Recursively compute MaxSubSum of right half • Compute MaxSubSum of sequence constrained to use Acenter and A(center+1) • Return max { left_max, right_max, center_max } • E.g., <1, -4, 4, 2, -3, 5, 8, -2> maxleft maxright maxcenter

  44. MaxSubSum: Solution 3 MaxSubSum3 (A, i, j) maxSum = 0 if (i = j) then if A[i] > 0 then maxSum = A[i] else k = floor((i+j)/2) maxSumLeft = MaxSubSum3(A,i,k) maxSumRight = MaxSubSum3(A,k+1,j) compute maxSumThruCenter maxSum = Maximum (maxSumLeft, maxSumRight, maxSumThruCenter) return maxSum

  45. // how to find the max that passes through the center Keep right end fixed at center and vary left end Keep left end fixed at center+1 and vary right end Add the two to determine max through center

  46. Solution 3: Analysis • T(1) = Θ(1) • T(N) = 2T(N/2) + Θ(N) • T(N) = Θ(?) Can we do even better? T(N) = (N log N)

  47. Observation Any negative subsequence cannot be a prefix to the maximum sequence Or, only a positive, contiguous subsequence is worth adding MaxSubSum: Solution 4 E.g., <1, -4, 4, 2, -3, 5, 8, -2> MaxSubSum4 (A) maxSum = 0 sum = 0 for j = 1 to N sum = sum + A[j] if (sum > maxSum) then maxSum = sum else if (sum < 0) then sum = 0 return maxSum Can we do even better? T(N) = (N)

More Related