Analysis design of algorithms csce 321
This presentation is the property of its rightful owner.
Sponsored Links
1 / 27

Analysis & Design of Algorithms (CSCE 321) PowerPoint PPT Presentation


  • 107 Views
  • Uploaded on
  • Presentation posted in: General

Analysis & Design of Algorithms (CSCE 321). Prof. Amr Goneid Department of Computer Science, AUC Part 2. Types of Complexities. Types of Complexities. Types of Complexities. Rules for Upper Bound Comparing Complexities Types of Complexities. 1. Rules for Upper Bound.

Download Presentation

Analysis & Design of Algorithms (CSCE 321)

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Analysis design of algorithms csce 321

Analysis & Design of Algorithms(CSCE 321)

Prof. Amr Goneid

Department of Computer Science, AUC

Part 2. Types of Complexities

Prof. Amr Goneid, AUC


Types of complexities

Types of Complexities

Prof. Amr Goneid, AUC


Types of complexities1

Types of Complexities

  • Rules for Upper Bound

  • Comparing Complexities

  • Types of Complexities

Prof. Amr Goneid, AUC


1 rules for upper bound

1. Rules for Upper Bound

If (k) is a constant, then:

  • O(k) O(n)

  • O(k f(n)) = O(f(n))

O(kn)

T(n)

O(n)

O(k)

n

Prof. Amr Goneid, AUC


Rules for big o

Rules for Big O

  • O(f(n)) + O(g(n)) = max(O(f(n)) ,O(g(n)))

  • e.g. f(n) = 2n = O(n), g(n) = 0.1 n3 = O(n3)

    T(n) = max(O(n) , O(n3)) = O(n3)

2n

O(n3)

0.1 n3

Prof. Amr Goneid, AUC


Rules for big o1

Rules for Big O

  • O(f(n)) * O(g(n)) = O(f(n) * g(n))

    e.g. f(n) = n, g(n) = n2

    T(n) = n * n2 = O(n3)

Repeat n times

O(n2)

O(n3)

Prof. Amr Goneid, AUC


Rules for big o2

Rules for Big O

  • O(log n)  O(n)Prove!

    Claim:

    for all n ≥ 1, log n ≤ n.

    The proof is by induction on n.

    The claim is trivially true for n = 1, since 0 < 1.

    Now suppose n ≥ 1 and log n ≤ n.

    Then,

    log(n + 1) ≤ log 2n

    = logn + 1

    ≤ n + 1 (by induction hypothesis)

Prof. Amr Goneid, AUC


Rules for big o3

Rules for Big O

  • For a polynomial of degree m,

  • Prove!

  • O(nm-1)  O(nm) follows from above

Prof. Amr Goneid, AUC


Rules for big o4

Rules for Big O

Prof. Amr Goneid, AUC


Summary of rules for big o

Summary of Rules for Big-O

Prof. Amr Goneid, AUC


2 comparing complexities

2. Comparing Complexities

  • Dominance:

    If lim(n->) f(n)/g(n) = 

    then f(n) dominates (i.e. grows faster), but if

    lim(n->) f(n)/g(n) = 0

    then g(n) dominates.

    In the latter case, we say that f(n) = o(g(n))

little oh

Prof. Amr Goneid, AUC


Comparing complexities

Comparing Complexities

Examples:

  • if a > b then na dominates nb since

    lim(n->) nb / na = 0

  • n2 dominates (3n+2) since

    lim(n->) (3n+2) / n2 = 0

  • n2 dominates (n log n) since

    lim(n->) (n log n) / n2 = lim(n->) log n / n = 0

Prof. Amr Goneid, AUC


Using l hopital s rule 1696

Using l’Hopital’s Rule (1696)

Example: Show that

f(n) = n(k+α) + nk (log n)2 and

g(n) = k n(k+α)

grow at the same rate.

Prof. Amr Goneid, AUC


Comparing complexities1

Comparing Complexities

  • Which grows faster:

    f(n) = n3 or g(n) = n log n

    f(n) = n 0.001 org(n) = log n

    f(n) = 2n+1 org(n) = 2n

    f(n) = 2n org(n) = 22n

Prof. Amr Goneid, AUC


Exercises

Exercises

Which function has smaller complexity ?

  • f = 100 n4g = n5

  • f = log(log n3)g = log n

  • f = n2g = n log n

  • f = 50 n5 + n2 + ng = n5

  • f = eng = n!

Prof. Amr Goneid, AUC


3 types of complexities

3. Types of Complexities

  • Constant Complexity

    • T(n) = constant independent of (n)

    • Runs in constant amount of time  O(1)

    • Example: cout << a[0][0]

  • Polynomial Complexity

    • T(n)=amnm+…+ a2n2 + a1n1 + a0

    • If m=1, then O(a1n+a0)  O(n)

    • If m > 1, then  O(nm) as nm dominates

Prof. Amr Goneid, AUC


Analysis design of algorithms csce 321

Polynomials Complexities

O(n3)

O(n2)

Log T(n)

O(n)

n

Prof. Amr Goneid, AUC


Types of complexities2

Types of Complexities

  • Logarithmic Complexity

    • Log2n=m is equivalent to n=2m

    • Reduces the problem to half  O(log2n)

    • Example: Binary Search

      T(n) = O(log2n)

      Much faster than Linear Search which has T(n) = O(n)

Prof. Amr Goneid, AUC


Analysis design of algorithms csce 321

Linear vs Logarithmic Complexities

T(n)

O(n)

O(log2n)

n

Prof. Amr Goneid, AUC


Types of complexities3

Types of Complexities

  • Exponential

    • Example: List all the subsets of a set of n elements {a,b,c}

      {a,b,c}, {a,b},{a,c},{b,c},{a},{b},{c},{}

    • Number of operations T(n) = O(2n)

    • Exponential expansion of the problem  O(an) where a is a constant greater than 1

Prof. Amr Goneid, AUC


Analysis design of algorithms csce 321

Exponential Vs Polynomial

O(2n)

Log T(n)

O(n3)

O(n)

n

Prof. Amr Goneid, AUC


Types of complexities4

Types of Complexities

  • Factorial time Algorithms

    • Example:

    • Traveling salesperson problem (TSP):

      Find the best route to take in visiting n cities away from home. What are the number of possible routes? For 3 cities: (A,B,C)

Prof. Amr Goneid, AUC


Possible routes in a tsp

Possible routes in a TSP

  • Number of operations = 3!=6, Hence T(n) = n!

  • Expansion of the problem  O(n!)

Prof. Amr Goneid, AUC


Analysis design of algorithms csce 321

Exponential Vs Factorial

O(nn)

O(n!)

Log T(n)

O(2n)

n

Prof. Amr Goneid, AUC


Execution time example

Execution Time Example

  • Example:

    • For the exponential algorithm of listing all subsets of a given set, assume the set size to be of 1024 elements

    • Number of operations is 21024 about 1.8*10308

    • If we can list a subset every nanosecond the process will take 5.7 * 10291 yr!!!

Prof. Amr Goneid, AUC


P and np times

P and NP – Times

  • P (Polynomial) Times:

    O(1), O(log n), O(log n)2, O(n), O(n log n),

    O(n2), O(n3), ….

  • NP (Non-Polynomial) Times:

    O(2n) , O(en) , O(n!) , O(nn) , …..

Prof. Amr Goneid, AUC


P and np times1

P and NP – Times

  • Polynomial time is

    GOOD

    Try to reduce the polynomial

    power

  • NP (e.g. Exponential) Time is

    BAD

Prof. Amr Goneid, AUC


  • Login