Algorithms
This presentation is the property of its rightful owner.
Sponsored Links
1 / 60

Algorithms PowerPoint PPT Presentation


  • 70 Views
  • Uploaded on
  • Presentation posted in: General

Algorithms. Asymptotic Performance. Review: Asymptotic Performance. Asymptotic performance : How does algorithm behave as the problem size gets very large? Running time Memory/storage requirements Remember that we use the RAM model: All memory equally expensive to access

Download Presentation

Algorithms

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Algorithms

Algorithms

Asymptotic Performance


Review asymptotic performance

Review: Asymptotic Performance

  • Asymptotic performance: How does algorithm behave as the problem size gets very large?

    • Running time

    • Memory/storage requirements

  • Remember that we use the RAM model:

    • All memory equally expensive to access

    • No concurrent operations

    • All reasonable instructions take unit time

      • Except, of course, function calls

    • Constant word size

      • Unless we are explicitly manipulating bits


Review running time

Review: Running Time

  • Number of primitive steps that are executed

    • Except for time of executing a function call most statements roughly require the same amount of time

    • We can be more exact if need be

  • Worst case vs. average case


An example insertion sort

An Example: Insertion Sort

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}


An example insertion sort1

An Example: Insertion Sort

i =  j = key = A[j] =  A[j+1] = 

30

10

40

20

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort2

An Example: Insertion Sort

i = 2j = 1key = 10A[j] = 30 A[j+1] = 10

30

10

40

20

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort3

An Example: Insertion Sort

i = 2j = 1key = 10A[j] = 30 A[j+1] = 30

30

30

40

20

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort4

An Example: Insertion Sort

i = 2j = 1key = 10A[j] = 30 A[j+1] = 30

30

30

40

20

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort5

An Example: Insertion Sort

i = 2j = 0key = 10A[j] =  A[j+1] = 30

30

30

40

20

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort6

An Example: Insertion Sort

i = 2j = 0key = 10A[j] =  A[j+1] = 30

30

30

40

20

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort7

An Example: Insertion Sort

i = 2j = 0key = 10A[j] =  A[j+1] = 10

10

30

40

20

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort8

An Example: Insertion Sort

i = 3j = 0key = 10A[j] =  A[j+1] = 10

10

30

40

20

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort9

An Example: Insertion Sort

i = 3j = 0key = 40A[j] =  A[j+1] = 10

10

30

40

20

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort10

An Example: Insertion Sort

i = 3j = 0key = 40A[j] =  A[j+1] = 10

10

30

40

20

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort11

An Example: Insertion Sort

i = 3j = 2key = 40A[j] = 30 A[j+1] = 40

10

30

40

20

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort12

An Example: Insertion Sort

i = 3j = 2key = 40A[j] = 30 A[j+1] = 40

10

30

40

20

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort13

An Example: Insertion Sort

i = 3j = 2key = 40A[j] = 30 A[j+1] = 40

10

30

40

20

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort14

An Example: Insertion Sort

i = 4j = 2key = 40A[j] = 30 A[j+1] = 40

10

30

40

20

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort15

An Example: Insertion Sort

i = 4j = 2key = 20A[j] = 30 A[j+1] = 40

10

30

40

20

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort16

An Example: Insertion Sort

i = 4j = 2key = 20A[j] = 30 A[j+1] = 40

10

30

40

20

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort17

An Example: Insertion Sort

i = 4j = 3key = 20A[j] = 40 A[j+1] = 20

10

30

40

20

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort18

An Example: Insertion Sort

i = 4j = 3key = 20A[j] = 40 A[j+1] = 20

10

30

40

20

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort19

An Example: Insertion Sort

i = 4j = 3key = 20A[j] = 40 A[j+1] = 40

10

30

40

40

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort20

An Example: Insertion Sort

i = 4j = 3key = 20A[j] = 40 A[j+1] = 40

10

30

40

40

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort21

An Example: Insertion Sort

i = 4j = 3key = 20A[j] = 40 A[j+1] = 40

10

30

40

40

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort22

An Example: Insertion Sort

i = 4j = 2key = 20A[j] = 30 A[j+1] = 40

10

30

40

40

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort23

An Example: Insertion Sort

i = 4j = 2key = 20A[j] = 30 A[j+1] = 40

10

30

40

40

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort24

An Example: Insertion Sort

i = 4j = 2key = 20A[j] = 30 A[j+1] = 30

10

30

30

40

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort25

An Example: Insertion Sort

i = 4j = 2key = 20A[j] = 30 A[j+1] = 30

10

30

30

40

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort26

An Example: Insertion Sort

i = 4j = 1key = 20A[j] = 10 A[j+1] = 30

10

30

30

40

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort27

An Example: Insertion Sort

i = 4j = 1key = 20A[j] = 10 A[j+1] = 30

10

30

30

40

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort28

An Example: Insertion Sort

i = 4j = 1key = 20A[j] = 10 A[j+1] = 20

10

20

30

40

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4


An example insertion sort29

An Example: Insertion Sort

i = 4j = 1key = 20A[j] = 10 A[j+1] = 20

10

20

30

40

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

1

2

3

4

Done!


Animating insertion sort

Animating Insertion Sort

  • Check out the Animator, a java applet at:http://www.cs.hope.edu/~alganim/animator/Animator.html

  • Try it out with random, ascending, and descending inputs


Insertion sort

Insertion Sort

What is the precondition

for this loop?

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}


Insertion sort1

Insertion Sort

InsertionSort(A, n) {for i = 2 to n {key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {A[j+1] = A[j]j = j - 1}A[j+1] = key}

}

How many times will this loop execute?


Insertion sort2

Insertion Sort

Statement Effort

InsertionSort(A, n) {

for i = 2 to n { c1n

key = A[i]c2(n-1)

j = i - 1;c3(n-1)

while (j > 0) and (A[j] > key) {c4T

A[j+1] = A[j]c5(T-(n-1))

j = j - 1c6(T-(n-1))

}0

A[j+1] = keyc7(n-1)

}0

}

T = t2 + t3 + … + tn where ti is number of while expression evaluations for the ith for loop iteration


Analyzing insertion sort

Analyzing Insertion Sort

  • T(n)=c1n + c2(n-1) + c3(n-1) + c4T + c5(T - (n-1)) + c6(T - (n-1)) + c7(n-1) = c8T + c9n + c10

  • What can T be?

    • Best case -- inner loop body never executed

      • ti = 1  T(n) is a linear function

    • Worst case -- inner loop body executed for all previous elements

      • ti = i  T(n) is a quadratic function

    • Average case

      • ???


Analysis

Analysis

  • Simplifications

    • Ignore actual and abstract statement costs

    • Order of growth is the interesting measure:

      • Highest-order term is what counts

        • Remember, we are doing asymptotic analysis

        • As the input size grows larger it is the high order term that dominates


Growth of functions

Growth of Functions

  • Asymptotic: Growth of running time of algorithm is relevant to growth of input size

  • Asymptotic Notations:

    • These are the functions with domains of natural numbers

  • Normally worst case running time is considered

  • These notations are abused but not misused


Asymptotic notations

Asymptotic Notations

  • Following are various asymptotic notations

    • Big O notation: i.e. O(g(n2))

    • Theta notation: i.e. (g(n))

    • Omega Notation: i.e. (n)

    • Small o notation: i.e. o(g(n2))


Big o notation asymptotic upper bound

Big O Notation (Asymptotic Upper Bound)

  • Asymptotic Upper Bound

  • In general a function

    • f(n) is O(g(n)) if there exist positive constants c and n0such that f(n)  c  g(n) for all n  n0

  • Formally

    • O(g(n)) = { f(n):  positive constants c and n0such that f(n)  c  g(n)  n  n0


Upper bound notation

Upper Bound Notation

  • We say Insertion Sort’s run time is O(n2)

    • Properly we should say run time is in O(n2)

    • Read O as “Big-O” (you’ll also hear it as “order”)


Insertion sort is o n 2

Insertion Sort Is O(n2)

  • Proof

    • Suppose runtime is an2 + bn + c

      • If any of a, b, and c are less than 0 replace the constant with its absolute value

    • an2 + bn + c  (a + b + c)n2 + (a + b + c)n + (a + b + c)

    •  3(a + b + c)n2 for n  1

    • Let c’ = 3(a + b + c) and let n0 = 1

  • Question

    • Is InsertionSort O(n3)?

    • Is InsertionSort O(n)?


Big o fact

Big O Fact

  • A polynomial of degree k is O(nk)

  • Proof:

    • Suppose f(n) = bknk + bk-1nk-1 + … + b1n + b0

      • Let ai = | bi |

    • f(n)  aknk + ak-1nk-1 + … + a1n + a0


Big o notation

Big O Notation

  • (g(n)) ≤ O (g(n))

  • Any quadratic function in (n2) is also in O(n2)

  • Any linear function an+b is also in O(n2)

  • It is normal to distinguish the asymptotic upper bound with asymptomatic tight bound.

  • O(g(n)) is applicable for every inputs whereas it is not the case for (g(n))


Big o notation1

Big O Notation


Asymptotic tight bound theta notation

Asymptotic Tight Bound: Theta Notation

  • A function f(n) is (g(n)) if  positive constants c1, c2, and n0 such that c1 g(n)  f(n)  c2 g(n)  n  n0

  • f(n) is asymptomatically tight bound for f(n)

  • Theorem

    • f(n) is (g(n)) iff f(n) is both O(g(n)) and (g(n))

    • Proof: someday


Theta notation

Theta Notation

  • Every member f(n) E g(n) must be non negative

  • Example: f(n)=(½)n2 – 3n = (n2)

    • Find c1, c2 and n0

    • C1(n2)  f(n)  C2(n2)

  • After calculation C1  1/14 , C2  ½

  • Other values of constants may be found depending upon the function


Theta notation1

Theta Notation

  • The function is sandwiched


Lower bound notation

Lower Bound Notation

  • We say InsertionSort’s run time is (n)

  • In general a function

    • f(n) is (g(n)) if  positive constants c and n0such that 0  cg(n)  f(n)  n  n0

  • Proof:

    • Suppose run time is an + b

      • Assume a and b are positive (what if b is negative?)

    • an  an + b

  • It purpose is to show that running time will always remain upper than given limit


Lower bound notation1

Lower Bound Notation


Theorem

Theorem

  • For any functions f(n) and g(n), we have f(n)= (g(n)) if and only if f(n)=O(g(n)) and f(n)=(g(n))

  • Proof:

    • An2+bn+c= (g(n2)) for any constant a,b,c where a>0


Practical complexity

Practical Complexity


Practical complexity1

Practical Complexity


Practical complexity2

Practical Complexity


Practical complexity3

Practical Complexity


Practical complexity4

Practical Complexity


Other asymptotic notations

Other Asymptotic Notations

  • A function f(n) is o(g(n)) if  positive constants c and n0such that f(n) < c g(n)  n  n0

  • A function f(n) is (g(n)) if  positive constants c and n0such that c g(n) < f(n)  n  n0

  • Intuitively,

  • o() is like <

  • O() is like 

  • () is like >

  • () is like 

  • () is like =


Up next

Up Next

  • Solving recurrences

    • Substitution method

    • Master theorem


  • Login