Chapter 23 algorithm efficiency
Download
1 / 103

chapter 23 algorithm efficiency - PowerPoint PPT Presentation


  • 394 Views
  • Updated On :

Chapter 23 Algorithm Efficiency. Objectives (1). To estimate algorithm efficiency using the Big O notation (§23.2) To understand growth rates and why constants and smaller terms can be ignored in the estimation (§23.2)

Related searches for chapter 23 algorithm efficiency

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'chapter 23 algorithm efficiency' - niveditha


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Objectives 1 l.jpg
Objectives (1)

  • To estimate algorithm efficiency using the Big O notation (§23.2)

  • To understand growth rates and why constants and smaller terms can be ignored in the estimation (§23.2)

  • To know the examples of algorithms with constant time, logarithmic time, linear time, log-linear time, quadratic time, and exponential time (§23.2)

  • To analyze linear search, binary search, selection sort, and insertion sort (§23.2)


Objectives 2 l.jpg
Objectives (2)

  • To design, implement, and analyze bubble sort (§23.3)

  • To design, implement, and analyze merge sort (§23.4)

  • To design, implement, and analyze quick sort (§23.5)

  • To design, implement, and analyze heap sort (§23.6)

  • To sort large data in a file (§23.7)


Analysis of algorithms l.jpg

Analysis of Algorithms

Input

Algorithm

Output

An algorithm is a step-by-step procedure for

solving a problem in a finite amount of time


Characterizing algorithms l.jpg
Characterizing Algorithms

  • Investigating the run times of algorithms and data structure operations

  • Focus will be on the relationship between running time of an algorithms and the size of the input


Can we write better algorithms l.jpg
Can We Write Better Algorithms?

“Better.”

―Michelangelo, when asked how he would have made his statue of Moses if he had to do it over again



Executing time question l.jpg
Executing Time Question

  • Suppose two algorithms perform the same task such as search (linear search vs. binary search) and sorting (selection sort vs. insertion sort)

    • Which one is better?

  • One possible approach to answer this question is to implement these algorithms in Java and run the programs to get execution time

  • But there are two problems with this approach…..


Problems measuring execution time l.jpg
Problems Measuring Execution Time

  • First, there are many tasks running concurrently on a computer

    • The execution time of a particular program is dependent on the system load

  • Second, the execution time is dependent on specific input

    • Consider linear search and binary search

      • If an element to be searched happens to be the first in the list, linear search will find the element quicker than binary search


Growth rate of running time l.jpg
Growth Rate of Running Time

  • Changing the hardware/ software environment

    • Affects running time by a constant factor

      • It but does not alter the growth rate


Experimental studies l.jpg
Experimental Studies

  • Write a program implementing an algorithm

  • Run the program with inputs of varying size and composition

  • Use a method like System.currentTimeMillis()to get an accurate measure of the actual running time

  • Plot the results


Limitations of experiments l.jpg
Limitations of Experiments

  • It is necessary to implement the algorithm, which may be difficult

  • Results may not be indicative of the running time on other inputs not included in the experiment

  • In order to compare two algorithms, the same hardware and software environments must be used


Theoretical analysis l.jpg
Theoretical Analysis

  • Uses a high-level description of the algorithm instead of an implementation

  • Characterizes running time as a function of the input size, n

  • Takes into account all possible inputs

  • Allows us to evaluate the speed of an algorithm independent of the hardware/software environment


Growth rate 1 l.jpg
Growth Rate (1)

  • It is very difficult to compare algorithms by measuring their execution time

  • To overcome these problems, a theoretical approach was developed to analyze algorithms independent of computers and specific input

  • This approach approximates the effect of a change on the size of the input


Growth rate 2 l.jpg
Growth Rate (2)

  • In this way, one can see how fast an algorithm’s execution time increases as the input size increases, so one can compare two algorithms by examining their growth rates


Execution time 1 l.jpg
Execution Time (1)

  • The linear search algorithm compares the key with the elements in the array sequentially until the key is found or the array is exhausted

    • If the key is not in the array, it requires n comparisons for an array of size n

    • If the key is in the array, it requires n/2 comparisons on average


Execution time 2 l.jpg
Execution Time (2)

  • The algorithm’s execution time is proportional to the size of the array

  • If one doubles the size of the array, one will expect the number of comparisons to double

  • The algorithm grows at a linear rate

  • The growth rate has an order of magnitude of n


Big o notation l.jpg
Big O Notation

  • Computer scientists use the Big O notation to abbreviate for “order of magnitude”

  • ” Using this notation, the complexity of the linear search algorithm is O(n),pronounced as “order of n”


Best worst and average cases l.jpg
Best, Worst, and Average Cases

  • For the same input size, an algorithm’s execution time may vary, depending on the input

    • An input that results in the shortest execution time is called the best-case input and an input that results in the longest execution time is called the worst-case input

  • Best-caseand worst-case are not representative, but worst-case analysis is very useful

  • One can show that the algorithm will never be slower than the worst-case

  • An average-case analysis attempts to determine the average amount of time among all possible input of the same size


Worse case l.jpg
Worse Case

  • Average-case analysis is ideal, but difficult to perform, because it is hard to determine the relative probabilities and distributions of various input instances for many problems

  • Worst-case analysis is easier to obtain and is thus common

    • Analysis is generally conducted for the worst-case

    • Crucial to applications such as games, finance and robotics


Useful formulas l.jpg
Useful Formulas

Arithmetic sum

Geometric sum


Summation notation l.jpg
Summation Notation

Polynomials


Seven functions l.jpg
Seven Functions

  • These functions are often used in algorithm analysis

    • Constant  1

    • Logarithmic  log n

    • Linear  n

    • n-log-n  n log n

    • Quadratic  n2

    • Cubic  n3

    • Exponential  2n


Constant function l.jpg
Constant Function

  • f (n) = c

    • Typically f (n) = 1 is used during algorithm analysis

    • A constant function is used to characterize the number of steps needed to do a basic operations

      • Time is not related to the input size

        • Adding two numbers

        • Assigning a value to a variable

        • Comparing two numbers

        • Retrieves an element at a given index in an array


Logarithm function l.jpg
Logarithm Function

  • f (n) = logbnfor some constant b

    • The function is defined as follows

      • x= logbnif and only ifbx=n

    • 2 is the most common base used during algorithms analysis

  • If one squares the input size, one only doubles the time for the algorithm of O(logn)


Basic properties of logarithms l.jpg
Basic Properties of Logarithms

logb(xy) = logbx + logby

logb (x/y) = logbx - logby

logbxa = alogbx

logba = logda/logdb

b logda = a logdb

log2n= log n/ log2 (converts base 10 to base 2)


Linear function l.jpg
Linear Function

  • f (n) = n

    • Function arises when the same basic operation is done for n elements

      • For example

        • Comparing a constant c to each element of an array of size n requires n comparisons

        • Reading n objects requires n operations


N log n function l.jpg
n-log-n Function

  • f (n) = n log2n

    • This function

      • Grows a little faster than the linear function, f (n) =n

      • Grows much slower than the quadratic n2 , f (n) = n2

    • If we can improve the running time of solving some problem from a quadratic to n-log-n, we will have an algorithm that runs much faster


Quadratic function l.jpg
Quadratic Function

  • f (n) = n2

    • This function is used in analysis since many algorithms contain nested loops

      • The inner loop performs n operations while the outer loop performs n operations (n * n = n2)


Quadratic time l.jpg
Quadratic Time

  • An algorithm with the O(n2) time complexity is called a quadratic algorithm

  • The quadratic algorithm grows quickly as the problem size increases

  • If one doubles the input size, the time for the algorithm is quadrupled

  • Algorithms with a nested loop are often quadratic


Nested loops and the quadratic function l.jpg
Nested Loops and the Quadratic Function

  • The quadratic function also is used in the context of nested loops where the first iteration uses one operation, the second uses two operations, the third uses three operations, and so on (for the inner loop)

    for (inti = 0; i < n; i++) {

    for (int j = 0; j < i; j++) {

    {

    System.out.print('*');

    }

    }


Nested loops and the quadratic function32 l.jpg
Nested Loops and the Quadratic Function

  • The number of operations performed is

  • 1 + 2 + 3+ … + (n-2) + (n-1) + n = n(n+1)/2=(n2+n)/2 =n2/2+n/2

  • Note: n2 >(n2+n)/2

  • An algorithm characterized where the first iteration uses one operation, the second uses two operations, the third uses three operations and so on, is slightly better than an algorithm that uses n operations each time through the loop (n2)


Cubic function l.jpg
Cubic Function

  • f (n) = n3

    • This function is used often used in algorithm analysis


Polynomials l.jpg
Polynomials

  • f (n) = ao +a1n +a2n2+… adnd

    • This function has degree d

    • Running times that are polynomials with degree d are generally better than polynomials with a larger degree

    • Note: the constant, linear, quadratic, and cubic functions are polynomials


Exponential function l.jpg
Exponential Function

  • f (n) = bn

    • Most common based used in algorithm analysis is 2

    • If a loops starts by performing one operation and then doubles the number of operations performed within each iteration

      • The number of operations performed in the nthiteration is 2n


Basic properties of exponentials l.jpg
Basic Properties of Exponentials

(ab)c = abc

abac = a(b+c)

ab/ac = a(b-c)


Comparison of functions l.jpg
Comparison of Functions

  • The slope of the line corresponds to the growth rate of the function


Big oh inequality l.jpg
Big-Oh Inequality

  • Let f(n)and g(n) be non-negative functions

    • Then f(n)is O(g(n)) if there are positive constant c and n0 1 such that f(n)cg(n) for n n0

    • This definition is referred to as the big-Ohnotation or f(n) is big-Oh of g(n) or f(n)is order of g(n)


Big oh and growth rate l.jpg
Big-Oh and Growth Rate

  • The big-Oh notation gives an upper bound on the growth rate of a function

  • The statement “f(n) is O(g(n))” means that the growth rate of f(n)is no more than the growth rate of g(n)


Ignoring multiplicative constants l.jpg
Ignoring Multiplicative Constants

  • The linear search algorithm requires n comparisons in the worst-case and n/2 comparisons in the average-case

    • Using the Big O notation, both cases require O(n)time

    • The multiplicative constant (1/2) is normally omitted

  • Algorithm analysis is focused on growth rate

  • The multiplicative constants have little impact on growth rates

  • The growth rates n/2 and 100n are equivalent to n

    • O(n) = O(n/2) = O(100n)


Ignoring non dominating terms 1 l.jpg
Ignoring Non-Dominating Terms (1)

  • Consider the algorithm for finding the maximum number in an array of n elements

    • If n is 2, it takes one comparison to find the maximum number

    • If n is 3, it takes two comparisons to find the maximum number

    • In general, it takes n-1 times of comparisons to find maximum number in a list of nelements

If the input size is small, there is no significance to estimate an algorithm’s efficiency


Ignoring non dominating terms 2 l.jpg
Ignoring Non-Dominating Terms (2)

  • Algorithm analysis is done for large input sizes

  • If is f(n) a polynomial of degree d, then f(n) is O(nd),

    • Drop lower-order terms

    • Drop constant factors

    • As n grows larger, the n part in the expression n-1dominates the complexity


Big oh rules l.jpg
Big-Oh Rules

  • Use the smallest possible class of functions

    • Say “2n is O(n)” instead of “2n is O(n2)”

  • Use the simplestexpression of the class

    • Say “3n + 5 is O(n)” instead of “3n + 5 is O(3n)”


Big oh example 1 l.jpg
Big-Oh Example (1)

  • 8n - 2 is O(n)

    • Justification

    • Find c and n0 such that 8n-2  cn

      • Pick c = 8 and n0 = 1 (there are an infinite number of solutions)

        8n - 2 8n


Big oh example 2 l.jpg
Big-Oh Example (2)

  • 2n +10 is O(n)

    • Justification

      2n + 10 cn

      2n-cn  -10

      n(2-c)  -10

      n(c-2)  10

      n  10/(c-2)

      Let c=3 and n0=10


Big oh examples 3 l.jpg
Big-Oh Examples (3)

  • 5n4+3n3+2n2+4n+1 is O(n4)

    • Justification

      • 5n4+3n3+2n2+4n+1  (5+3+2+4+1) n4 =cn4 where c=15 and n0=1

  • 5n2+3log(n)+2n+5 is O(n2)

    • Justification

      • 5n2+3log(n)+2n+5  (5+3+2+5) n2=cn2 where c=15 and nn0=1


Big oh examples 4 l.jpg
Big-Oh Examples (4)

  • 20n3+10nlog(n)+5 is O(n3)

    • Justification

      • 20n3+10nlog(n)+5  35 n3 =cn3 where c=35 and nn0=2

  • 3log(n)+2 is O(log n)

    • Justification

      • 3log(n)+2  5 log(n)where c=5 and n2


Big oh examples 5 l.jpg
Big-Oh Examples (5)

  • 2n+1is O(2n)

    • Justification

      • 2n+1 = 2n *21 =2* 2n 2* 2n where c=2 and n0=1

  • 2n+100log(n) is O(n)

    • Justification

      • 2n+100log(n)  102nwhere c=102 and n n0=2

  • 3 log n + 5 is O(log n)

    • Justification

      • 3 log n + 5 8 log n where c = 8 and n0 = 2


Big oh example 6 l.jpg
Big-Oh Example (6)

  • n2is not O(n)

    • Justification

      n2cn

      n c

    • The above inequality cannot be satisfied since c must be a constant


Increasing common growth functions l.jpg
Increasing Common Growth Functions

Constant time

Logarithmic time

Linear time

Log-linear time

Quadratic time

Cubic time

Exponential time


Pseudo code l.jpg
Pseudo-code

  • High-level description of an algorithm

  • More structured than English prose

  • Less detailed than a program

  • Preferred notation for describing algorithms

  • Hides program design issues


Primitive operations l.jpg

Basic computations performed by an algorithm

Identifiable in pseudo-code

Largely independent from the programming language

Exact definition not important

Example primitive operations:

Evaluating an expression

Assigning a value to a variable

Indexing into an array

Calling a method

Returning from a method

Primitive Operations


Counting primitive operations l.jpg
Counting Primitive Operations

  • Primitive operations correspond to a low level instruction

    • Assumption is made that execution time is the same constant forall primitives

    • Primitive operations are counted to measure execution time


Pseudo code details l.jpg

Control flow

if … then … [else …]

while … do …

repeat … until …

for … do …

Indentation replaces braces

Method declaration

Algorithm method (arg [, arg…])

Input …

Output …

Method call

var.method(arg [, arg…])

Return value

returnexpression

Expressions

Assignment(like  in Java)

Equality testing(like  in Java)

n2 Superscripts and other mathematical formatting allowed

Pseudo-code Details


Asymptotic algorithm analysis l.jpg
Asymptotic Algorithm Analysis

  • The asymptotic analysis of an algorithm determines the running time in big-O notation

  • To perform the asymptotic analysis

    • We find the worst-case number of primitive operations executed as a function of the input size

      • Average case analysis requires sophisticated probability analysis

    • We express this function with big-O notation

  • Since constant factors and lower-order terms are eventually dropped anyhow, we can disregard them when counting primitive operations


Determining big oh l.jpg
Determining Big-Oh

  • Repetition

  • Sequence

  • Selection

  • Logarithm


Repetition simple loops l.jpg

constant time

executed

n times

Repetition: Simple Loops

for (i = 1; i <= n; i++) {

k = k + 5;

}

Time Complexity

T(n) = (a constant c) * n = cn = O(n)

Ignore multiplicative constants (e.g., “c”).

c is the time to execute a simple statement


Repetition nested loops 1 l.jpg

constant time

inner loop

executed

n times

executed

n times

Repetition: Nested Loops (1)

for (i = 1; i <= n; i++) {

for (j = 1; j <= n; j++) {

k = k + i + j;

}

}

Time Complexity

T(n) = (a constant c) * n * n = cn2 = O(n2)

Ignore multiplicative constants (e.g., “c”).


Repetition nested loops 2 l.jpg

constant time

inner loop

executed

i times

executed

n times

Repetition: Nested Loops (2)

for (i = 1; i <= n; i++) {

for (j = 1; j <= i; j++) {

k = k + i + j;

}

}

Time Complexity

T(n) = c + 2c + 3c + 4c + … + nc = cn(n+1)/2 = (c/2)n2 + (c/2)n = O(n2)

Ignore non-dominating terms

Ignore multiplicative constants


Repetition nested loops 3 l.jpg

constant time

inner loop

executed

20 times

executed

n times

Repetition: Nested Loops (3)

for (i = 1; i <= n; i++) {

for (j = 1; j <= 20; j++) {

k = k + i + j;

}

}

Time Complexity

T(n) = 20 * c * n = O(n)

Ignore multiplicative constants (e.g., 20*c)


Sequence l.jpg

inner loop

executed

20 times

executed

n times

executed

10 times

Sequence

for (j = 1; j <= 10; j++) {

k = k + 4;

}

for (i = 1; i <= n; i++) {

for (j = 1; j <= 20; j++) {

k = k + i + j;

}

}

Time Complexity

T(n) = c *10 + 20 * c * n = O(n)


Selection l.jpg

Executed

n times

Selection

if (list.contains(e)) {

System.out.println(e);

}

else

for (Object t: list) {

System.out.println(t);

}

O(n)

Time Complexity

T(n) = test time + worst-case (if, else)

= O(n) + O(n)

= O(n)


Slide63 l.jpg

Prefix Averages (Quadratic)

The following algorithm computes prefix averages in quadratic time by applying the definition

AlgorithmprefixAverages1(X, n)

Input array X of n integers

Output array A of prefix averages of X #operations

A new array of n integers n

fori 0 ton 1 do n

s  X[0] n

forj 1 toido 1 + 2 + …+ (n 1)

s  s+X[j] 1 + 2 + …+ (n 1)

A[i]  s/(i+ 1)n

returnA 1

Since the sum of the first n integers is n(n + 1) / 2, the running time is4n+1+n2+n2 isO(n2)


Slide64 l.jpg

Prefix Averages (Linear)

The following algorithm computes prefix averages in linear time by keeping a running sum

AlgorithmprefixAverages2(X, n)

Input array X of n integers

Output array A of prefix averages of X #operations

A new array of n integers n

s  0 1

fori 0 ton 1 do n

s  s+X[i] n

A[i]  s/(i+ 1)n

returnA 1

Algorithm prefixAverages2 runs in O(n) time


Find maximum element of an array using pseudo code l.jpg
Find Maximum Element of an Array using Pseudo-code

AlgorithmarrayMax(A, n)

Input array A of n integers

currentMax A[0]

fori 1 ton  1 do

ifA[i]  currentMaxthen

currentMax  A[i]

returncurrentMax

Output maximum element of A

What is the Big O representation of this algorithm?


Linear search l.jpg
Linear Search

  • Consider the set {0,1,2,3,4,5}

  • One would start from the index of the set, 0, and then look at each next value until it finds the value a particular value

    • It would take four comparisons looking for 3

    • It would take six comparison looking for 6

  • From this one concludes that the algorithm will make a number of comparisons equal to the input size, n, and thus is O(n)


Binary search 1 l.jpg
Binary Search (1)

  • For binary search to work, the elements in the array must already be ordered

  • Without loss of generality, assume that the array is in ascending order

    e.g., 2 4 7 10 11 45 50 59 60 66 69 70 79

  • The binary search first compares the key with the element in the middle of the array


Binary search 2 l.jpg
Binary Search (2)

Consider the following three cases:

  • If the key is less than the middle element, one only needs to search the key in the first half of the array

  • If the key is equal to the middle element, the search ends with a match

  • If the key is greater than the middle element, one only needs to search the key in the second half of the array


Binary search l.jpg
Binary Search

Key

List

8

8

8




Binary search results l.jpg
Binary Search Results

  • The binarySearchmethod returns the index of the element in the list that matches the search key if it is contained in the list

  • Otherwise, it returns insertion point – 1


Binary search algorithm l.jpg
Binary Search Algorithm

public class BinarySearch {

/** Use binary search to find the key in the list */

public static int binarySearch(int[] list, int key) {

int low = 0;

int high = list.length - 1;

while (high >= low) { // the number of times through the loop determines Big O

int mid = (low + high) / 2;

if (key < list[mid])

high = mid - 1;

else if (key == list[mid])

return mid;

else

low = mid + 1;

}

return -low - 1; // Now high < low

}

}


Analyzing the binary search l.jpg
Analyzing the Binary Search

  • If the original number of items is N then after the first iteration there will be at most N/2 items remaining, then at most N/4 items, at most N/8 items, and so on

  • In the worst case, when the value is not in the list, the algorithm must continue iterating until the span has been made empty; this will have taken at most log2(N) + 1 iterations

    • For example, if N=16, there are at most 5 iterations needed (log2(16)+1=5 )


Linear search vs binary search l.jpg
Linear Search vs. Binary Search

  • When compared to linear search whose worst-case behavior is N iterations, the binary search is substantially faster as N grows large

    • For example, to search a list of one million items takes as many as one million iterations with linear search, but never more than twenty iterations with binary search

    • However, a binary search can only be performed if the list is in sorted order


Selection sort l.jpg
Selection Sort

  • Selection sort finds the largest number in the list and places it last. It then finds the largest number remaining and places it next to last, and so on until the list contains only a single number


Selection sort77 l.jpg
Selection Sort

int[] myList = {2, 9, 5, 4, 8, 1, 6}; // Unsorted


Selection sort algorithm l.jpg
Selection Sort Algorithm

public class SelectionSort {

/** The method for sorting the numbers *

public static void selectionSort(double[] list) {

for (int i = list.length - 1; i >= 1; i--) {

// Find the maximum in the list[0..i]

double currentMax = list[0];

int currentMaxIndex = 0;

for (int j = 1; j <= i; j++) {

if (currentMax < list[j]) {

currentMax = list[j];

currentMaxIndex = j;

}

}

// Swap list[i] with list[currentMaxIndex] if necessary;

if (currentMaxIndex != i) {

list[currentMaxIndex] = list[i];

list[i] = currentMax;

}

}

}


Analyzing selection sort l.jpg
Analyzing Selection Sort

  • The selection sort algorithm finds the largest number in the list and places it last

    • It then finds the largest number remaining and places it next to last, and so on until the list contains only a single number

    • The number of comparisons is n-1 for the first iteration, n-2 for the second iteration, and so on

  • Let T(n) denote the complexity for selection sort and c denote the total number of other operations such as assignments and additional comparisons in each iteration (see arithmetic sum)

  • Ignoring constants and smaller terms, the complexity of the selection sort algorithm is O(n2)


Insertion sort l.jpg
Insertion Sort

The insertion sort algorithm sorts a list of values by repeatedly inserting an unsorted element into a sorted sublist until the whole list is sorted.

int[] myList = {2, 9, 5, 4, 8, 1, 6}; // Unsorted


Insertion sort81 l.jpg
Insertion Sort

int[] myList = {2, 9, 5, 4, 8, 1, 6}; // Unsorted


Insertion algorithm l.jpg
Insertion Algorithm

public class InsertionSort {

/** The method for sorting the numbers */

public static void insertionSort(double[] list) {

for (int i = 1; i < list.length; i++) {

/** insert list[i] into a sorted sublist list[0..i-1] so that

list[0..i] is sorted. */

double currentElement = list[i];

int k;

for (k = i - 1; k >= 0 && list[k] > currentElement; k--) {

list[k + 1] = list[k];

}

// Insert the current element into list[k+1]

list[k + 1] = currentElement;

}

}

}

}


Analyzing insertion sort l.jpg
Analyzing Insertion Sort

  • The insertion sort algorithm sorts a list of values by repeatedly inserting a new element into a sorted partial array until the whole array is sorted

    • At the kth iteration, to insert an element to a array of size k, it may take k comparisons to find the insertion position, and k moves to insert the element

  • Let T(n) denote the complexity for insertion sort and c denote the total number of other operations such as assignments and additional comparisons in each iteration

  • Ignoring constants and smaller terms, the complexity of the insertion sort algorithm is O(n2)


Fibonacci numbers l.jpg
Fibonacci Numbers

0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987,…

The next number in the sequence is the sum of the previous two numbers


Fibonacci numbers recursion l.jpg
Fibonacci Numbers Recursion

/** The method for finding the Fibonacci number */

public static long fib(long index) {

if (index == 0) // Base case

return 0;

else if (index == 1) // Base case

return 1;

else // Reduction and recursive calls

return fib(index - 1) + fib(index - 2);

}


Complexity for recursive fibonacci numbers l.jpg
Complexity for Recursive Fibonacci Numbers

Since

and

The recursive Fibonacci method takes O(2n)


More efficient method to calculate fibonacci numbers l.jpg
More Efficient Method to Calculate Fibonacci Numbers

  • Avoid repeated call to the fib method with the same arguments

  • The next Fibonacci number is calculated by adding the preceding two numbers in the sequence

  • If one stores the values of the preceding two numbers (not always calculating them), the algorithm is much more efficient


Non recursive version of fibonacci numbers l.jpg
Non-recursive version of Fibonacci Numbers

/** The method for finding the Fibonacci number */

public static long fib(long n) {

if (n == 0 || n == 1)

return n;

long f0 = 0; // For fib(0)

long f1 = 1; // For fib(1)

long currentFib = 1; // For fib(2)

for (int i = 2; i <= n; i++) {

currentFib = f0 + f1;

f0 = f1;

f1 = currentFib;

}

return currentFib;

}

  • The complexity of this new algorithm is O(n)

  • This is a tremendous improvement over the recursive algorithm


Practical considerations l.jpg
Practical Considerations

  • The big O notation provides a good theoretical estimate of algorithm efficiency

  • However, two algorithms of the same time complexity are not necessarily equally efficient

  • Even though the algorithms run in O(n) time, in terms of practically, the last one is more efficient than the previous one



Greatest common divisor gcd algorithm version 1 l.jpg
Greatest Common Divisor (GCD) Algorithm Version 1

GCD is the largest number that evenly divides both integers

public static int gcd(int m, int n) {

int gcd = 1;

for (int k = 2; k <= m && k <= n; k++) {

if (m % k == 0 && n % k == 0)

gcd = k;

}

return gcd;

}

This method checks whether k is a common divisor for m and n until k > m or n

The complexity of this algorithm is O(n)


Gcd algorithms version 2 l.jpg
GCD Algorithms Version 2

public static int gcd(int m, int n) {

int gcd = 1;

for (int k = n; k >= 1; k--) {

if (m % k == 0 && n % k == 0) {

gcd = k;

break;

}

return gcd;

}

It is more efficient to search from n down

Once is divisor is found, the algorithm is complete

The worst-case time complexity of this algorithm is O(n)


Gcd algorithms version 3 l.jpg
GCD Algorithms Version 3

Assuming m  n, the loop is executed at most n/2 times which cuts the time in half

public static int gcd(int m, int n) {

int gcd = 1;

if (m == n) return m;

for (int k = n / 2; k >= 1; k--) {

if (m % k == 0 && n % k == 0) {

gcd = k;

break;

}

return gcd;

}

The worst-case time complexity of this algorithm is O(n)


Euclid s algorithm l.jpg
Euclid’s Algorithm

  • If m % n is 0, gcd (m, n) is n

  • Otherwise, gcd(m, n) is equal to gcd(n, m % n)

    • Proof

    • Suppose m % n = r

    • Then m=q*n +r where q is the quotient of m/n

    • Any number divisible by m and n must also be divisible by r

    • Hence the gcd (n, m) is the same as gcd (n, r) where r = m % n


Euclid s algorithm examples l.jpg
Euclid’s Algorithm Examples

  • If m % n is 0, gcd (m, n) is n

    • Let m = 6 and n=3 then 6 % 3 = 0 and gcd (6, 3) = 3

  • Otherwise, gcd(m, n) is equal to gcd(n, m % n)

    • Let m = 100 and n=16 then gcd (100 ,16) = gcd (16, 4)

    • Because 100 % 16 = 4

    • Note: 100 =6*16 +4


Euclid s algorithm implementation l.jpg
Euclid’s Algorithm Implementation

public static int gcd(int m, int n) {

if (m % n == 0)

return n;

else

return gcd(n, m % n);

}


Euclid s algorithm analysis 1 l.jpg
Euclid’s Algorithm Analysis (1)

  • Assuming m  n, then m % n < m / 2

  • Euclid’s algorithm recursively calls the gcd method

    • 1st call: gcd (m, n)

    • 2nd call: gcd (n, m % n)

    • 3rd call: gcd (m % n, n % (m % n))

    • 4th call: gcd (n % (m % n), (m % n) % (n % (m % n)))


Euclid s algorithm analysis 2 l.jpg
Euclid’s Algorithm Analysis (2)

  • m % n < m / 2 and n % (m % n) < n /2

  • The argument passed to the gcd method is reduced by half after each iteration

  • Let k be the number of times the gcd method is called, the second parameter is less than which is greater of equal to 1

The time complexity of this algorithm is O(logn)


Finding prime numbers l.jpg
Finding Prime Numbers

  • Brute-force

  • Check possible divisors up to Math.sqrt(n)

  • Check possible prime divisors up to Math.sqrt(n)


Brute force method l.jpg
Brute-force Method

  • Find the first fifty primes

  • For each number n, look for divisors between 2 and n / 2

The time complexity of this algorithm is O(n)

PrimeNumber

Run


Math sqrt n method l.jpg
Math.sqrt(n) Method

  • This methods checks for divisors between 2 and the integer part of square root of n

  • for (int divisor = 2; divisor <= (int)(Math.sqrt(number));

  • divisor++) {

  • if number % divisor == 0) { // If true, number is not prime

  • isPrime = false; // Set isPrime to false

  • break; // Exit the for loop

  • }

  • }

The time complexity of this algorithm is


Prime divisors up to math sqrt n l.jpg
Prime Divisors up to Math.sqrt(n)

int squareRoot = 1; // Check whether number <= squareRoot

System.out.println("The prime numbers are \n");

// Repeatedly find prime numbers

while (number <= n) {

// Assume the number is prime

boolean isPrime = true; // Is the current number prime?

if (squareRoot * squareRoot < number) squareRoot++; //

// ClosestPair if number is prime

for (int k = 0; k < list.size()

&& list.get(k) <= squareRoot; k++) {

if (number % list.get(k) == 0) { // If true, not prime

isPrime = false; // Set isPrime to false

break; // Exit the for loop

}

}


Prime divisors up to math sqrt n103 l.jpg
Prime Divisors up to Math.sqrt(n)

  • There is not necessary to calculate (int)(Math.sqrt(number)) during each iteration

    • One needs to look at only perfect squares (4, 9, 16, 25,...)

    • For example, (int)(Math.sqrt(number)) is 6 for all the numbers between 36 and 48

    • SquareRoot is only incremented when squareRoot * squareRoot < number

    • The algorithm takes steps

Since the time complexity is


ad