- 120 Views
- Uploaded on
- Presentation posted in: General

Analysis & Design of Algorithms (CSCE 321)

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Analysis & Design of Algorithms(CSCE 321)

Prof. Amr Goneid

Department of Computer Science, AUC

Part 2. Types of Complexities

Prof. Amr Goneid, AUC

Prof. Amr Goneid, AUC

- Rules for Upper Bound
- Comparing Complexities
- Types of Complexities

Prof. Amr Goneid, AUC

If (k) is a constant, then:

- O(k) O(n)
- O(k f(n)) = O(f(n))

O(kn)

T(n)

O(n)

O(k)

n

Prof. Amr Goneid, AUC

- O(f(n)) + O(g(n)) = max(O(f(n)) ,O(g(n)))
- e.g. f(n) = 2n = O(n), g(n) = 0.1 n3 = O(n3)
T(n) = max(O(n) , O(n3)) = O(n3)

2n

O(n3)

0.1 n3

Prof. Amr Goneid, AUC

- O(f(n)) * O(g(n)) = O(f(n) * g(n))
e.g. f(n) = n, g(n) = n2

T(n) = n * n2 = O(n3)

Repeat n times

O(n2)

O(n3)

Prof. Amr Goneid, AUC

- O(log n) O(n)Prove!
Claim:

for all n ≥ 1, log n ≤ n.

The proof is by induction on n.

The claim is trivially true for n = 1, since 0 < 1.

Now suppose n ≥ 1 and log n ≤ n.

Then,

log(n + 1) ≤ log 2n

= logn + 1

≤ n + 1 (by induction hypothesis)

Prof. Amr Goneid, AUC

- For a polynomial of degree m,
- Prove!
- O(nm-1) O(nm) follows from above

Prof. Amr Goneid, AUC

Prof. Amr Goneid, AUC

Prof. Amr Goneid, AUC

- Dominance:
If lim(n->) f(n)/g(n) =

then f(n) dominates (i.e. grows faster), but if

lim(n->) f(n)/g(n) = 0

then g(n) dominates.

In the latter case, we say that f(n) = o(g(n))

little oh

Prof. Amr Goneid, AUC

Examples:

- if a > b then na dominates nb since
lim(n->) nb / na = 0

- n2 dominates (3n+2) since
lim(n->) (3n+2) / n2 = 0

- n2 dominates (n log n) since
lim(n->) (n log n) / n2 = lim(n->) log n / n = 0

Prof. Amr Goneid, AUC

Example: Show that

f(n) = n(k+α) + nk (log n)2 and

g(n) = k n(k+α)

grow at the same rate.

Prof. Amr Goneid, AUC

- Which grows faster:
f(n) = n3 or g(n) = n log n

f(n) = n 0.001 org(n) = log n

f(n) = 2n+1 org(n) = 2n

f(n) = 2n org(n) = 22n

Prof. Amr Goneid, AUC

Which function has smaller complexity ?

- f = 100 n4g = n5
- f = log(log n3)g = log n
- f = n2g = n log n
- f = 50 n5 + n2 + ng = n5
- f = eng = n!

Prof. Amr Goneid, AUC

- Constant Complexity
- T(n) = constant independent of (n)
- Runs in constant amount of time O(1)
- Example: cout << a[0][0]

- Polynomial Complexity
- T(n)=amnm+…+ a2n2 + a1n1 + a0
- If m=1, then O(a1n+a0) O(n)
- If m > 1, then O(nm) as nm dominates

Prof. Amr Goneid, AUC

Polynomials Complexities

O(n3)

O(n2)

Log T(n)

O(n)

n

Prof. Amr Goneid, AUC

- Logarithmic Complexity
- Log2n=m is equivalent to n=2m
- Reduces the problem to half O(log2n)
- Example: Binary Search
T(n) = O(log2n)

Much faster than Linear Search which has T(n) = O(n)

Prof. Amr Goneid, AUC

Linear vs Logarithmic Complexities

T(n)

O(n)

O(log2n)

n

Prof. Amr Goneid, AUC

- Exponential
- Example: List all the subsets of a set of n elements {a,b,c}
{a,b,c}, {a,b},{a,c},{b,c},{a},{b},{c},{}

- Number of operations T(n) = O(2n)
- Exponential expansion of the problem O(an) where a is a constant greater than 1

- Example: List all the subsets of a set of n elements {a,b,c}

Prof. Amr Goneid, AUC

Exponential Vs Polynomial

O(2n)

Log T(n)

O(n3)

O(n)

n

Prof. Amr Goneid, AUC

- Factorial time Algorithms
- Example:
- Traveling salesperson problem (TSP):
Find the best route to take in visiting n cities away from home. What are the number of possible routes? For 3 cities: (A,B,C)

Prof. Amr Goneid, AUC

- Number of operations = 3!=6, Hence T(n) = n!
- Expansion of the problem O(n!)

Prof. Amr Goneid, AUC

Exponential Vs Factorial

O(nn)

O(n!)

Log T(n)

O(2n)

n

Prof. Amr Goneid, AUC

- Example:
- For the exponential algorithm of listing all subsets of a given set, assume the set size to be of 1024 elements
- Number of operations is 21024 about 1.8*10308
- If we can list a subset every nanosecond the process will take 5.7 * 10291 yr!!!

Prof. Amr Goneid, AUC

- P (Polynomial) Times:
O(1), O(log n), O(log n)2, O(n), O(n log n),

O(n2), O(n3), ….

- NP (Non-Polynomial) Times:
O(2n) , O(en) , O(n!) , O(nn) , …..

Prof. Amr Goneid, AUC

- Polynomial time is
GOOD

Try to reduce the polynomial

power

- NP (e.g. Exponential) Time is
BAD

Prof. Amr Goneid, AUC