algorithms
Download
Skip this Video
Download Presentation
Algorithms

Loading in 2 Seconds...

play fullscreen
1 / 60

Algorithms - PowerPoint PPT Presentation


  • 97 Views
  • Uploaded on

Algorithms. Asymptotic Performance. Review: Asymptotic Performance. Asymptotic performance : How does algorithm behave as the problem size gets very large? Running time Memory/storage requirements Remember that we use the RAM model: All memory equally expensive to access

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Algorithms' - lani-atkinson


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
algorithms

Algorithms

Asymptotic Performance

review asymptotic performance
Review: Asymptotic Performance
  • Asymptotic performance: How does algorithm behave as the problem size gets very large?
      • Running time
      • Memory/storage requirements
    • Remember that we use the RAM model:
      • All memory equally expensive to access
      • No concurrent operations
      • All reasonable instructions take unit time
        • Except, of course, function calls
      • Constant word size
        • Unless we are explicitly manipulating bits
review running time
Review: Running Time
  • Number of primitive steps that are executed
    • Except for time of executing a function call most statements roughly require the same amount of time
    • We can be more exact if need be
  • Worst case vs. average case
an example insertion sort
An Example: Insertion Sort

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

an example insertion sort1
An Example: Insertion Sort

i =  j =  key = A[j] =  A[j+1] = 

30

10

40

20

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort2
An Example: Insertion Sort

i = 2 j = 1 key = 10A[j] = 30 A[j+1] = 10

30

10

40

20

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort3
An Example: Insertion Sort

i = 2 j = 1 key = 10A[j] = 30 A[j+1] = 30

30

30

40

20

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort4
An Example: Insertion Sort

i = 2 j = 1 key = 10A[j] = 30 A[j+1] = 30

30

30

40

20

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort5
An Example: Insertion Sort

i = 2 j = 0 key = 10A[j] =  A[j+1] = 30

30

30

40

20

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort6
An Example: Insertion Sort

i = 2 j = 0 key = 10A[j] =  A[j+1] = 30

30

30

40

20

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort7
An Example: Insertion Sort

i = 2 j = 0 key = 10A[j] =  A[j+1] = 10

10

30

40

20

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort8
An Example: Insertion Sort

i = 3 j = 0 key = 10A[j] =  A[j+1] = 10

10

30

40

20

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort9
An Example: Insertion Sort

i = 3 j = 0 key = 40A[j] =  A[j+1] = 10

10

30

40

20

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort10
An Example: Insertion Sort

i = 3 j = 0 key = 40A[j] =  A[j+1] = 10

10

30

40

20

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort11
An Example: Insertion Sort

i = 3 j = 2 key = 40A[j] = 30 A[j+1] = 40

10

30

40

20

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort12
An Example: Insertion Sort

i = 3 j = 2 key = 40A[j] = 30 A[j+1] = 40

10

30

40

20

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort13
An Example: Insertion Sort

i = 3 j = 2 key = 40A[j] = 30 A[j+1] = 40

10

30

40

20

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort14
An Example: Insertion Sort

i = 4 j = 2 key = 40A[j] = 30 A[j+1] = 40

10

30

40

20

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort15
An Example: Insertion Sort

i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 40

10

30

40

20

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort16
An Example: Insertion Sort

i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 40

10

30

40

20

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort17
An Example: Insertion Sort

i = 4 j = 3 key = 20A[j] = 40 A[j+1] = 20

10

30

40

20

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort18
An Example: Insertion Sort

i = 4 j = 3 key = 20A[j] = 40 A[j+1] = 20

10

30

40

20

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort19
An Example: Insertion Sort

i = 4 j = 3 key = 20A[j] = 40 A[j+1] = 40

10

30

40

40

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort20
An Example: Insertion Sort

i = 4 j = 3 key = 20A[j] = 40 A[j+1] = 40

10

30

40

40

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort21
An Example: Insertion Sort

i = 4 j = 3 key = 20A[j] = 40 A[j+1] = 40

10

30

40

40

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort22
An Example: Insertion Sort

i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 40

10

30

40

40

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort23
An Example: Insertion Sort

i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 40

10

30

40

40

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort24
An Example: Insertion Sort

i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 30

10

30

30

40

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort25
An Example: Insertion Sort

i = 4 j = 2 key = 20A[j] = 30 A[j+1] = 30

10

30

30

40

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort26
An Example: Insertion Sort

i = 4 j = 1 key = 20A[j] = 10 A[j+1] = 30

10

30

30

40

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort27
An Example: Insertion Sort

i = 4 j = 1 key = 20A[j] = 10 A[j+1] = 30

10

30

30

40

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort28
An Example: Insertion Sort

i = 4 j = 1 key = 20A[j] = 10 A[j+1] = 20

10

20

30

40

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

an example insertion sort29
An Example: Insertion Sort

i = 4 j = 1 key = 20A[j] = 10 A[j+1] = 20

10

20

30

40

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

1

2

3

4

Done!

animating insertion sort
Animating Insertion Sort
  • Check out the Animator, a java applet at:http://www.cs.hope.edu/~alganim/animator/Animator.html
  • Try it out with random, ascending, and descending inputs
insertion sort
Insertion Sort

What is the precondition

for this loop?

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

insertion sort1
Insertion Sort

InsertionSort(A, n) {for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key}

}

How many times will this loop execute?

insertion sort2
Insertion Sort

Statement Effort

InsertionSort(A, n) {

for i = 2 to n { c1n

key = A[i] c2(n-1)

j = i - 1; c3(n-1)

while (j > 0) and (A[j] > key) { c4T

A[j+1] = A[j] c5(T-(n-1))

j = j - 1 c6(T-(n-1))

} 0

A[j+1] = key c7(n-1)

} 0

}

T = t2 + t3 + … + tn where ti is number of while expression evaluations for the ith for loop iteration

analyzing insertion sort
Analyzing Insertion Sort
  • T(n)=c1n + c2(n-1) + c3(n-1) + c4T + c5(T - (n-1)) + c6(T - (n-1)) + c7(n-1) = c8T + c9n + c10
  • What can T be?
    • Best case -- inner loop body never executed
      • ti = 1  T(n) is a linear function
    • Worst case -- inner loop body executed for all previous elements
      • ti = i  T(n) is a quadratic function
    • Average case
      • ???
analysis
Analysis
  • Simplifications
    • Ignore actual and abstract statement costs
    • Order of growth is the interesting measure:
      • Highest-order term is what counts
        • Remember, we are doing asymptotic analysis
        • As the input size grows larger it is the high order term that dominates
growth of functions
Growth of Functions
  • Asymptotic: Growth of running time of algorithm is relevant to growth of input size
  • Asymptotic Notations:
    • These are the functions with domains of natural numbers
  • Normally worst case running time is considered
  • These notations are abused but not misused
asymptotic notations
Asymptotic Notations
  • Following are various asymptotic notations
    • Big O notation: i.e. O(g(n2))
    • Theta notation: i.e. (g(n))
    • Omega Notation: i.e. (n)
    • Small o notation: i.e. o(g(n2))
big o notation asymptotic upper bound
Big O Notation (Asymptotic Upper Bound)
  • Asymptotic Upper Bound
  • In general a function
    • f(n) is O(g(n)) if there exist positive constants c and n0such that f(n)  c  g(n) for all n  n0
  • Formally
    • O(g(n)) = { f(n):  positive constants c and n0such that f(n)  c  g(n)  n  n0
upper bound notation
Upper Bound Notation
  • We say Insertion Sort’s run time is O(n2)
    • Properly we should say run time is in O(n2)
    • Read O as “Big-O” (you’ll also hear it as “order”)
insertion sort is o n 2
Insertion Sort Is O(n2)
  • Proof
    • Suppose runtime is an2 + bn + c
      • If any of a, b, and c are less than 0 replace the constant with its absolute value
    • an2 + bn + c  (a + b + c)n2 + (a + b + c)n + (a + b + c)
    •  3(a + b + c)n2 for n  1
    • Let c’ = 3(a + b + c) and let n0 = 1
  • Question
    • Is InsertionSort O(n3)?
    • Is InsertionSort O(n)?
big o fact
Big O Fact
  • A polynomial of degree k is O(nk)
  • Proof:
    • Suppose f(n) = bknk + bk-1nk-1 + … + b1n + b0
      • Let ai = | bi |
    • f(n)  aknk + ak-1nk-1 + … + a1n + a0
big o notation
Big O Notation
  • (g(n)) ≤ O (g(n))
  • Any quadratic function in (n2) is also in O(n2)
  • Any linear function an+b is also in O(n2)
  • It is normal to distinguish the asymptotic upper bound with asymptomatic tight bound.
  • O(g(n)) is applicable for every inputs whereas it is not the case for (g(n))
asymptotic tight bound theta notation
Asymptotic Tight Bound: Theta Notation
  • A function f(n) is (g(n)) if  positive constants c1, c2, and n0 such that c1 g(n)  f(n)  c2 g(n)  n  n0
  • f(n) is asymptomatically tight bound for f(n)
  • Theorem
    • f(n) is (g(n)) iff f(n) is both O(g(n)) and (g(n))
    • Proof: someday
theta notation
Theta Notation
  • Every member f(n) E g(n) must be non negative
  • Example: f(n)=(½)n2 – 3n = (n2)
    • Find c1, c2 and n0
    • C1(n2)  f(n)  C2(n2)
  • After calculation C1  1/14 , C2  ½
  • Other values of constants may be found depending upon the function
theta notation1
Theta Notation
  • The function is sandwiched
lower bound notation
Lower Bound Notation
  • We say InsertionSort’s run time is (n)
  • In general a function
    • f(n) is (g(n)) if  positive constants c and n0such that 0  cg(n)  f(n)  n  n0
  • Proof:
    • Suppose run time is an + b
      • Assume a and b are positive (what if b is negative?)
    • an  an + b
  • It purpose is to show that running time will always remain upper than given limit
theorem
Theorem
  • For any functions f(n) and g(n), we have f(n)= (g(n)) if and only if f(n)=O(g(n)) and f(n)=(g(n))
  • Proof:
    • An2+bn+c= (g(n2)) for any constant a,b,c where a>0
other asymptotic notations
Other Asymptotic Notations
  • A function f(n) is o(g(n)) if  positive constants c and n0such that f(n) < c g(n)  n  n0
  • A function f(n) is (g(n)) if  positive constants c and n0such that c g(n) < f(n)  n  n0
  • Intuitively,
  • o() is like <
  • O() is like 
  • () is like >
  • () is like 
  • () is like =
up next
Up Next
  • Solving recurrences
    • Substitution method
    • Master theorem
ad