Algorithm Analysis

1 / 33

# Algorithm Analysis - PowerPoint PPT Presentation

Algorithm Analysis. Chris Kiekintveld CS 2401 (Fall 2010) Elementary Data Structures and Algorithms. Algorithm Analysis. There are many different algorithms to solve the same problem Ask 5 programmers to write a non-trivial program, you will get 5 different solutions Which is best ?

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Algorithm Analysis' - max

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Algorithm Analysis

Chris Kiekintveld

CS 2401 (Fall 2010)

Elementary Data Structures and Algorithms

Algorithm Analysis
• There are many different algorithms to solve the same problem
• Ask 5 programmers to write a non-trivial program, you will get 5 different solutions
• Which is best?
• Correctness
• Efficiency

Computational Resources
• Algorithms require resources to run
• Time (processor operations)
• Space (computer memory)
• Network bandwidth
• Programmer time
• Two types of costs
• Fixed: same every time we run the algorithm
• Variable: depends on the size of the input

Measuring Resource Use
• How can we compare the resources used by different algorithms?
• Empirical
• Code both algorithms
• Run them an record the resources used
• You did this in the Fibonacci lab!

Empirical Analysis Problems
• Depends on code quality/implementation
• Better/worse programmers, not the algorithm itself
• Depends on computer speed/architecture
• Depends on language/compiler efficiency
• Depends on the input
• E.g. linear search is very fast for some inputs, and very slow for others

Analytical Approach
• Analyze the algorithm itself
• Abstract away from implementation details
• How many operations will be executed?
• How much memory is used?
• Consider different cases (depending on input)
• Best
• Worst
• Average

Counting Operations

int i = 2;

int j = 2;

int k = i + j;

System.out.println(k);

How many operations are there?

Assignment:

Print:

Total:

Counting Operations

int i = 2;

int j = 2;

int k = i + j;

System.out.println(i+j+k);

How many operations are there?

Assignment:

Print:

Total:

Counting Operations

int i = 0;

while (i < 10) {

System.out.println(i);

i++;

}

How many operations are there?

Assignment:

Comparison:

Increment:

Print:

Total:

Counting Operations

for (int i=0; i < 10; i++) {

System.out.println(i);

}

How many operations are there?

Assignment:

Comparison:

Increment:

Print:

Total:

Counting Operations

for (int i=0; i < n; i++) {

System.out.println(i);

}

How many operations are there?

Assignment:

Comparison:

Increment:

Print:

Total:

Counting Operations

for (int i=0; i < n; i++) {

for (int j=0; j < n; j++) {

System.out.println(i);

}

}

How many operations are there?

Assignment:

Comparison:

Increment:

Print:

Total:

Counting Operations
• So far, we have counted every operation
• This is quite tedious, especially for infrequent operations
• Focus on the most important operation
• Most frequent
• May need to figure out what this is

Another Look at Search Algorithms
• We have discussed two ways to search a list
• Linear search (unordered data)
• Binary search (sorted data)
• Data is sorted by “keys”
• Unique for each element
• Well-defined order

Linear (Sequential) Search

public int seqSearch(T[] list, int length, T searchItem)

{

int loc;

boolean found = false;

for (loc = 0; loc < length; loc++)

{

if (list[loc].equals(searchItem))

{

found = true;

break;

}

}

if (found)

return loc;

else

return -1;

}

Sequential Search Analysis
• The statements in the forloop are repeated several times
• For each iteration of the loop, the search item is compared with an element in the list
• When analyzing a search algorithm, you count the number of comparisons
• Suppose that L is a list of length n
• The number of key comparisons depends on where in the list the search item is located

Sequential Search Analysis (continued)
• Best case
• The item is the first element of the list
• You make only one key comparison
• Worst case
• The item is the last element of the list
• You make n key comparisons
• What is the average case

Sequential Search Analysis (continued)
• To determine the average case
• Consider all possible cases
• Find the number of comparisons for each case
• Add them and divide by the number of cases
• Average case
• On average, a successful sequential search searches half the list

Binary Search

public int binarySearch(T[] list, int length, T searchItem)

{

int first = 0;

int last = length - 1;

int mid = -1;

boolean found = false;

while (first <= last && !found)

{

mid = (first + last) / 2;

Comparable<T> compElem = (Comparable<T>) list[mid];

Binary Search (continued)

if (compElem.compareTo(searchItem) == 0)

found = true;

else

if (compElem.compareTo(searchItem) > 0)

last = mid - 1;

else

first = mid + 1;

}

if (found)

return mid;

else

return -1;

}//end binarySearch

Binary Search Example

Figure 18-1 Sorted list for a binary search

Table 18-1 Values of first, last, and middle and the Number of

Comparisons for Search Item 89

Performance of Binary Search
• Suppose that L is a sorted list of size n
• And n is a power of 2 (n = 2m)
• After each iteration of the forloop, about half the elements are left to search
• The maximum number of iteration of the for loop is about m + 1
• Also m = log2n
• Each iteration makes two key comparisons
• Maximum number of comparisons: 2(m + 1)

Comparison: Linear vs BinaryWorst case number of comparison

Asymptotic Analysis: Motivation
• So far, we have counted operations exactly
• We don’t really care about the details
• Computers execute billions of operations per second
• A few here or there is negligible
• As the input size grows, does computation grow quickly or slowly?
• Don’t lose the forest for the trees

Asymptotic Analysis
• Asymptotic means the study of the function f as n becomes larger and larger without bound
• Consider functions g(n) = n2 and f(n) = n2 + 4n + 20
• As n becomes larger and larger, the term 4n + 20 in f(n) becomes insignificant
• g(1000) = 1,000,000 and f(1000) = 1,004,020
• You can predict the behavior of f(n) by looking at the behavior of g(n)

Asymptotic Algorithm Analysis
• Identify a function that describes the growth in runtime as the input gets large
• An “upper bound” of sorts on the running time
• Typically worst-case, but occasionally average case
• Describe the number of operations done using a function
• Focus only on most important operations
• Ignore one time initializations, etc.

Common Asymptotic Functions

Table 18-4 Growth Rate of Various Functions

Common Functions, visual

Figure 18-9 Growth Rate of Various Functions

Asymptotic Notation: Big-O Notation (continued)

Table 18-7 Some Big-O Functions That Appear in Algorithm Analysis

Big-Oh Notation (Definition)

A function f(n) is O(g(n)) if there exist positive constants c and n0 such that:

f(n) ≤ cg(n) for all n ≥ n0

Big-Oh Notation
• Translation: After some point, f(n) is always smaller than g(n)
• “Some point” refers to increasing problem size
• The constant c says that we don’t care about multipliers
• So, 2n and n have the same essential growth rate
• 2n is O(n)