Time Complexity

1 / 20

# Time Complexity - PowerPoint PPT Presentation

Time Complexity. The best, worst, and average-case complexities of a given algorithm are numerical functions of the size of the instances.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Time Complexity' - tymon

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Time Complexity

The best, worst, and average-case complexities of a given algorithm are numerical functions of the size of the instances.

It is difficult to work with these functions exactly because they are often very complicated, with many little up and down bumps. Thus it is usually cleaner and easier to talk about upper and lower bounds of such functions.

This is where the big Oh notation comes into the picture.

Chapter 2: Algorithm Analysis

Time Complexity

Upper and lower bounds smooth out the behavior of complex functions

Chapter 2: Algorithm Analysis

Time Complexity - Big-O

• T(n) = O(f(n))
• means c.f(n) is an upper bound on T(n), where there
• exists some constant c such that T(n) is always <= c.f(n) for large enough n.
• Example: n3 + 3n2 + 6n + 5 is O(n3).
• (Use c = 15 and n0 = 1.)
• Example: n2 + n logn is O(n2).
• (Use c = 2 and n0 = 1.)

Chapter 2: Algorithm Analysis

ALGORITHM

A

B

10

1,110

11,110

Input

Size

n

100

1,010,100

2,010,100

1,000

1,001,001,000

1,101,001,000

10,000

1,000,100,010,000

1,010,100,010,000

100,000

1,000,010,000,100,000

1,001,010,000,100,000

1,000,000

1,000,001,000,001,000,000

1,000,101,000,001,000,000

Demonstrating The Big-O Concept

Each of the algorithms below has O(n3) time complexity...

(In fact, the execution time for Algorithm A is n3 + n2 + n, and the execution time for Algorithm B is n3 + 101n2 + n.)

Chapter 2: Algorithm Analysis

ALGORITHM

C

D

10

123

10,123

Input

Size

n

100

10,203

110,203

1,000

1,002,003

2,002,003

10,000

100,020,003

110,020,003

100,000

10,000,200,003

10,100,200,003

1,000,000

1,000,002,000,003

1,001,002,000,003

A Second Big-O Demonstration

Each of the algorithms below has O(n2) time complexity...

(In fact, the execution time for Algorithm C is n2 + 2n + 3, and the execution time for Algorithm D is n2 + 1002n + 3.)

Chapter 2: Algorithm Analysis

ALGORITHM

E

F

10

83

1,083

Input

Size

n

100

1,164

11,164

1,000

14,966

114,966

10,000

182,877

1,182,877

100,000

2,160,964

12,160,964

1,000,000

24,931,569

124,931,569

One More Big-O Demonstration

Each of the algorithms below has O(nlogn) time complexity…

(In fact, the execution time for Algorithm E is n logn + 5n, and the execution time for Algorithm F is n logn + 105n. Note that the linear term for Algorithm F will dominate until n reaches 2105.)

Chapter 2: Algorithm Analysis

g(n)

v(n)

r(n)

p(n)

y(n)

b(n)

Big-O Represents An Upper Bound

If T(n) is O(f(n)), then f(n) is basically a cap on how bad T(n) will behave when n gets big.

YES!

YES!

YES!

Is g(n) O(r(n))?

Is v(n) O(y(n))?

Is b(n) O(p(n))?

YES!

YES!

NO!

Is r(n) O(g(n))?

Is y(n) O(v(n))?

Is p(n) O(b(n))?

Chapter 2: Algorithm Analysis

g(n)

r(n)

nr

Time Complexity Terminology: Big-Omega

• Function T(n) is said to be (g(n)) if there are positive constants c and n0 such that T(n)  c g (n) for any n  n0(i.e., T(n) is ultimately bounded below by c g (n)).
• Example: n3 + 3n2 + 6n + 5 is (n3). (Use c = 1 and n0 = 1.)
• Example: n2 + n logn is (n2). (Use c = 1 and n0 = 1.)

r(n) is not(g(n)) since for every positive constant c, (c)g(n) ultimately gets bigger than r(n)

g(n) is (r(n)) since g(n) exceeds (1)r(n) for all n-values past nr

Chapter 2: Algorithm Analysis

r(n)

g(n)

n0

Time Complexity Terminology: Big-Theta

• Function T(n) is said to be (h(n)) if T(n) is both O(h(n)) and (h(n)).
• Example: n3 + 3n2 + 6n + 5 is (n3).
• Example: n2 + n logn is (n2).

r(n) is (g(n)) since r(n) is squeezed between (1)g(n) and (2)g(n) once n exceeds n0

g(n) is (r(n)) since g(n) is squeezed between (½)r(n) and (1)r(n) once n exceeds n0

Chapter 2: Algorithm Analysis

Time Complexity Terminology: Little-O

• Function T(n) is said to be o(p (n)) if T(n) is O(p (n)) but not(p (n)).
• Example: n3 + 3n2 + 6n + 5 is O(n4). (Use c = 15 and n0 = 1.) However, n3 + 3n2 + 6n + 5 is not (n4).

Assume that there are positive constants c and n0 such that n3 + 3n2 + 6n + 5  c n4 for all n  n0.

Then dividing by n4 on both sides yields the fact that (1/n)+(3/n2)+(6/n3)+(5/n4)  c, for all n  n0.

Since limn((1/n)+(3/n2)+(6/n3)+(5/n4)) = 0, we must conclude that 0  c, which contradicts the fact that c must be a positive constant.

Chapter 2: Algorithm Analysis

Computational Model For Algorithm Analysis

To formally analyze the performance of algorithms, we will use a computational model with a couple of simplifying assumptions:

• Each simple instruction (assignment, comparison, addition, multiplication, memory access, etc.) is assumed to execute in a single time unit.
• Memory is assumed to be limitless, so there is always room to store whatever data is needed.

The size of the input, n, will normally be used as our main variable, and we’ll primarily be interested in “worst case” scenarios.

Chapter 2: Algorithm Analysis

General Rules For Running Time Calculation

Rule One: Loops

The running time of a loop is at most the running time of the statements inside the loop, multiplied by the number of iterations.

Example:

for (i = 0; i < n; i++) // n iterations

A[i] = (1-t)*X[i] + t*Y[i]; // 12 time units

// per iteration

(Retrieving X[i] requires one addition and one memory access, as does retrieving Y[i]; the calculation involves a subtraction, two multiplications, and an addition; assigning A[i] requires one addition and one memory access; and each loop iteration requires a comparison and either an assignment or an increment, thus totals twelve primitive operations.)

Thus, the total running time is 12n time units, i.e., this part of the program is O(n).

Chapter 2: Algorithm Analysis

Rule Two: Nested Loops

The running time of a nested loop is at most the running time of the statements inside the innermost loop, multiplied by the product of the number of iterations of all of the loops.

Example:

for (i = 0; i < n; i++) // n iterations. 2 ops each

for (j = 0; j < n; j++) // n iterations, 2 ops each

C[i,j] = j*A[i] + i*B[j]; // 10 time units/iteration

(2 for retrieving A[i], 2 for retrieving B[j], 3 for the RHS arithmetic, 3 for assigning C[i,j].)

Total running time: ((10+2)n+2)n = 12n2+2n time units, which is O(n2).

More complex example (ignoring for loop time):

for (i = 0; i < n; i++) // n iterations

for (j = i; j < n; j++) // n-i iterations

C[j,i] = C[i,j] = j*A[i]+i*B[j]; // 13 time units/iter

Total running time:  i=0,n-1( j=i, n-113) =  i=0,n-1(13(n-i))

= 13( i=0,n-1n -  i=0,n-1i) = 13(n2 - ½n(n-1))

= 6.5n2 + 6.5n time units, which is also O(n2).

Chapter 2: Algorithm Analysis

Rule Three: Consecutive Statements

The running time of a sequence of statements is merely the sum of the running times of the individual statements.

• Example:
• for (i = 0; i < n; i++)
• { // 22n time units
• A[i] = (1-t)*X[i] + t*Y[i]; // for this
• B[i] = (1-s)*X[i] + s*Y[i]; // entire loop
• }
• for (i = 0; i < n; i++) // (12n+2)n time
• for (j = 0; j < n; j++) // units for this
• C[i,j] = j*A[i] + i*B[j]; // nested loop

Total running time: 12n2+24n time units, i.e., this code is O(n2).

Chapter 2: Algorithm Analysis

Rule Four: Conditional Statements

The running time of an if-else statement is at most the running time of the conditional test, added to the maximum of the running times of the if and else blocks of statements.

Example:

if (amt > cost + tax) //2 time units

{

count = 0; //1 time unit

while ((count<n) && (amt>cost+tax)) //4 TUs per iter

{ //At most n iter

amt -= (cost + tax); //3 time units

count++; //2 time units

}

cout << “CAPACITY:” << count; //2 time units

}

else

cout << “INSUFFICIENT FUNDS”; //1 time unit

Total running time: 2 + max(1 + (4 + 3 + 2)n + 2, 1) = 9n + 5 time units, i.e., this code is O(n).

Chapter 2: Algorithm Analysis

Complete Analysis Of Binary Search Function

int binsrch(const etype A[], const etype x, const int n)

{

int low = 0, high = n-1; // 3 time units

int middle; // 0 time units

while (low <= high) // 1 time unit

{

middle = (low + high)/2; // 3 time units

if (A[middle] < x) // 2 TU | <-- Worst Case

low = middle + 1; // 2 TU |

else if (A[middle] > x) // 2 TU | <-- Worst Case

high = middle - 1; // 2 TU | <-- Worst Case

else // 0 TU |

return middle; // 1 TU |

}

return -1; // If search is unsuccessful; 1 time unit.

}

In the worst case, the loop will keep dividing the distance between the low and high indices in half until they are equal, iterating at most logn times. Thus, the total running time is: 10logn + 4 time units, which is O(logn).

Chapter 2: Algorithm Analysis

Analysis Of Another Function:SuperFreq

etype SuperFreq(const etype A[], const int n)

{

etype bestElement = A[0]; // 3 time units

int bestFreq = 0; // 1 time unit

int currFreq; // 0 time units

for (i = 0; i < n; i++) // n iterations; 2 TUs each

{

currFreq = 0; // 1 time unit

for (j = i; j < n; j++) // n-i iterations; 2 TUs each

if (A[i] == A[j]) // 3 time units

currFreq++; // 2 time units

if (currFreq > bestFreq) // 1 time unit

bestElement = A[i]; // 3 time units

}

return bestElement; // 1 time unit

}

Note that the function is obviously O(n2) due to its familiar nested loop structure. Specifically, its worst-case running time is ½(7n2 + 21n + 10).

Chapter 2: Algorithm Analysis

humongInt pow(const humongInt &val, const humongInt &n)

{

if (n == 0)

return humongInt(0);

if (n == 1)

return val;

if (n % 2 == 0)

return pow(val*val, n/2);

return pow(val*val, n/2) * val;

}

The worst-case running time would require all 3 conditions to be checked, and to fail (taking 4 time units).

The last return statement requires 3 time units each time it’s executed, which happens logn times (since it halves n with each execution, until it reaches a value of 1).

When the parameterized n-value finally reaches 1, two last operations are performed.

Thus, the worst-case running time is 7logn + 2.

Chapter 2: Algorithm Analysis

Recurrence Relations To Evaluate Recursion

int powerOf2(const int &n)

{

if (n == 0)

return 1;

return powerOf2(n-1) + powerOf2(n-1);

}

Assume that there is a function T(n) such that it takes T(k) time to executepowerOf2(k). Examining the code allows us to conclude the following:

T(0) = 2

T(k) = 5 + 2T(k-1) for all k > 0

The second fact tells us that:

T(n) = 5 + 2T(n-1) = 5 + 2(5 + 2T(n-2)) =

5 + 2(5 + 2(5 + 2(T(n-3)))) = …

= 5(1 + 2 + 22 + 23 + … + 2n-1) + 2nT(0) =

5(2n-1) + 2n(2) = 7(2n) - 5, which is O(2n).

Chapter 2: Algorithm Analysis

Another Recurrence Relation Example

int alternatePowerOf2(const int &n)

{

if (n == 0)

return 1;

return 2*alternatePowerOf2(n-1);

}

Assume that there is a function T(n) such that it takes T(k) time to executealternatePowerOf2(k). Examining the code allows us to conclude the following:

T(0) = 2

T(k) = 4 + T(k-1) for all k > 0

The second fact tells us that:

T(n) = 4 + T(n-1) = 4 + (4 + T(n-2)) =

+ (4 + (4 + (T(n-3)))) = …

= 4n + T(0) = 4n + 2, which is O(n).

Chapter 2: Algorithm Analysis