Comp108 algorithmic foundations algorithm efficiency searching sorting
Download
1 / 93

comp108 algorithmic foundations algorithm efficiency searching - PowerPoint PPT Presentation


  • 280 Views
  • Updated On :

COMP108 Algorithmic Foundations Algorithm efficiency + Searching/Sorting. Prudence Wong http://www.csc.liv.ac.uk/~pwong/teaching/comp108. Learning outcomes. Able to carry out simple asymptotic analysis of algorithms See some examples of polynomial time and exponential time algorithms

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'comp108 algorithmic foundations algorithm efficiency searching' - Gabriel


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Comp108 algorithmic foundations algorithm efficiency searching sorting l.jpg

COMP108Algorithmic FoundationsAlgorithm efficiency + Searching/Sorting

Prudence Wong

http://www.csc.liv.ac.uk/~pwong/teaching/comp108


Learning outcomes l.jpg
Learning outcomes

  • Able to carry out simple asymptotic analysis of algorithms

  • See some examples of polynomial time and exponential time algorithms

    • Able to apply searching/sorting algorithms and derive their time complexities


Time complexity analysis l.jpg
Time Complexity Analysis

  • How fast is the algorithm?

Code the algorithm and run the program, then measure the running time

  • Depend on the speed of the computer

  • Waste time coding and testing if the algorithm is slow

Identify some important operations/steps and count how many times these operations/steps needed to executed


Time complexity analysis4 l.jpg
Time Complexity Analysis

  • How to measure efficiency?

Number of operations usually expressed in terms of input size

  • If we doubled the input size, how much longer would the algorithm take?If we trebled the input size, how much longer would it take?


Why efficiency matters l.jpg
Why efficiency matters?

  • speed of computation by hardware has been improved

  • efficiency still matters

  • ambition for computer applications grow with computer power

  • demand a great increase in speed of computation


Amount of data handled matches speed increase l.jpg
Amount of data handled matches speed increase?

  • When computation speed vastly increased, can we handle much more data?

  • Suppose

    • an algorithm takes n2comparisons to sortn numbers

    • we need 1 sec to sort 5 numbers (25 comparisons)

    • computing speed increases by factor of100

  • Using 1 sec, we can now perform100x25comparisons,i.e., to sort50 numbers

  • With 100 times speedup, only sort 10 times more numbers!


Time complexity analysis7 l.jpg
Time Complexity Analysis

input n

sum = 0

for i = 1 to n do

begin

sum = sum + i

end

output sum

  • Important operation of summation: addition

  • How many additions this algorithm requires?

We need n additions

(depend on the input size n)


Space complexity analysis l.jpg
Space Complexity Analysis

input n

sum = 0

for i = 1 to n do

begin

sum = sum + i

end

output sum

We need to store the number n, the variable sum, and the index i

=> needs 3 memory space

In this example, the memory space does not depend on the input size n

In other case, it may depend on the input size n


Look for improvement l.jpg
Look for improvement

input n

sum = n*(n+1)/2

output sum

Mathematical formula gives us an alternative to find the sum of first n numbers:

1 + 2 + ... + n = n(n+1)/2

We only need 3 operations:

1 addition, 1 multiplication, and 1 division

(no matter what the input size n is)



Finding min among 3 numbers l.jpg
Finding min among 3 numbers

Input: 3 numbers a, b, c

Output: the min value of these 3 numbers

Algorithm:

Input a, b, c

a  b ?

N

Y

a  c ?

Y

b  c ?

N

Y

N

Output a

Output b

Output c

Output c


Finding min among 3 numbers12 l.jpg
Finding min among 3 numbers

Example 1

Input a, b, c

a=3, b=5, c=2

Y

N

a  b ?

a  c ?

Y

b  c ?

N

Y

N

Output a

Output b

Output c

Output c

2 is output


Finding min among 3 numbers13 l.jpg
Finding min among 3 numbers

Example 2

Input a, b, c

a=6, b=5, c=7

Y

N

a  b ?

a  c ?

Y

b  c ?

N

Y

N

Output a

Output b

Output c

Output c

5 is output


Time complexity analysis14 l.jpg
Time Complexity Analysis

  • Important operation: comparison

  • How many comparison this algorithm requires?

input a, b, c

if (a  b) then

if (a  c) then

output a

else

output c

else

if (b  c) then

output b

else

output c

It takes 2 comparisons


Finding min among n numbers l.jpg
Finding min among n numbers

Input a0, a1, ...

Any systematic approach?

a0  a1 ?

N

Y

a0  a2 ?

Y

a1  a2 ?

Y

N

N

a0  a3 ?

a1  a3 ?

a2  a3 ?

a2  a3 ?


Finding min among n numbers16 l.jpg
Finding min among n numbers

Input a[0], a[1], ..., a[n-1]

min = 0

i = 1

i = i+1

i < n?

N

Y

Output a[min]

a[i] < a[min]?

Y

min = i

N


Finding min among n numbers17 l.jpg
Finding min among n numbers

input a[0], a[1], ..., a[n-1]

min = 0

i = 1

while (i <n) do

begin

if (a[i] < a[min]) then

min = i

i = i + 1

end

output a[min]

Example

a[]={50,30,40,20,10}

0

50

1

30

1

30

3

20

4

10


Finding min among n numbers18 l.jpg
Finding min among n numbers

n-1

  • Time complexity: ?? comparisons

input a[0], a[1], ..., a[n-1]

min = 0

i = 1

while (i <n) do

begin

if (a[i] < a[min]) then

min = i

i = i + 1

end

output a[min]


Finding min using for loop l.jpg
Finding min using for-loop

  • Rewrite the above while-loop into a for-loop

input a[0], a[1], ..., a[n-1]

min = 0

for i = 1 to n-1 do

begin

if (a[i] < a[min]) then

min = i

end

output a[min]


Time complexity big o notation l.jpg

Time complexity- Big O notation …


Which algorithm is the fastest l.jpg
Which algorithm is the fastest?

  • Consider a problem that can be solved by 5 algorithms A1, A2, A3, A4, A5 using different number of operations (time complexity).

f1(n) = 50n + 20

f2(n) = 10 n log2 n + 100

f3(n) = n2 - 3n + 6

f4(n) = 2n2

f5(n) = 2n/8 - n/4 + 2

f5(n)

f3(n)

f1(n)

Depends on the size of the problem!


Slide22 l.jpg

f1(n) = 50n + 20

f2(n) = 10 n log2n + 100

f3(n) = n2 - 3n + 6

f4(n) = 2n2

f5(n) = 2n/8 - n/4 + 2


What do we observe l.jpg
What do we observe?

  • There is huge difference between

    • functions involving powers ofn (e.g., n, n log n, n2, called polynomial functions) and

    • functions involving powering by n (e.g., 2n, called exponential functions)

  • Among polynomial functions, those with same order of power are more comparable

    • e.g., f3(n) = n2 - 3n + 6 and f4(n) = 2n2



Relative growth rate l.jpg
Relative growth rate

2n

n2

n

log n

c

n


Hierarchy of functions l.jpg
Hierarchy of functions

  • We can define a hierarchy of functions each having a greater order of magnitude than its predecessor:

1

log n

n n2 n3 ... nk ...

2n

constant

logarithmic

polynomial

exponential

  • We can further refine the hierarchy by inserting n log n between n and n2,n2 log n between n2 and n3, and so on.


Hierarchy of functions 2 l.jpg
Hierarchy of functions (2)

1

log n

n n2 n3 ... nk ...

2n

  • Important: as we move from left to right, successive functions have greater order of magnitude than the previous ones.

  • As n increases, the values of the later functions increase more rapidly than the values of the earlier ones.

  • Relative growth rates increase

constant

logarithmic

polynomial

exponential


Hierarchy of functions 3 l.jpg
Hierarchy of functions (3)

1

log n

n n2 n3 ... nk ...

2n

  • Now, when we have a function, we can assign the function to some function in the hierarchy:

    • For example, f(n) = 2n3 + 5n2 + 4n + 7The term with the highest power is 2n3.The growth rate of f(n) is dominated by n3.

  • This concept is captured by Big-O notation

constant

logarithmic

polynomial

exponential


Big o notation l.jpg
Big-O notation

  • f(n) = O(g(n)) [read as f(n) is of order g(n)]

  • Roughly speaking, this means f(n) is at most a constant times g(n) for all large n

  • Examples

    • 2n3 = O(n3)

    • 3n2 = O(n2)

    • 2n log n = O(n log n)

    • n3 + n2 = O(n3)

  • function on L.H.S and function on R.H.S are said to have the same order of magnitude


Big o notation formal definition l.jpg
Big-O notation - formal definition

  • f(n) = O(g(n))

  • There exists a constant c and no such that f(n) cg(n) for all n > no

    • cnon>no then f(n)  cg(n)


F n o g n graphical illustration l.jpg
f(n) = O(g(n) – Graphical illustration

constant c & no such thatn>no, f(n)  c g(n)

c g(n)

f(n)

n0


Example n 60 o n l.jpg
Example – n+60 = O(n)

2n

n + 60

n

c=2, n0=60

n0

constant c & no such that n>no, f(n)  c g(n)


Example 2nlog 2 n 100 n 1000 o n log n l.jpg
Example – 2nlog2n+100n+1000 = O(n log n)

3 n log n

c=3, n0=600

f(n)

n log n

n0

constant c & no such that n>no, f(n)  c g(n)


F n o g n l.jpg
f(n) = O(g(n))?

c g(n)

f(n)

value of f(n) for n<n0does NOT matter

n0

constant c & no such that n>no, f(n)  c g(n)


Which one is the fastest l.jpg
Which one is the fastest?

  • Usually we are only interested in the asymptotic time complexity, i.e., when n is large

  • O(log n) < O(n) < O(n) < O(n log n) < O(n2) < O(2n)


Proof of order of magnitude l.jpg
Proof of order of magnitude

  • Show that 2n2 + 4n is O(n2)

    • Since n  n2 for all n>1,we have 2n2 + 4n  2n2 + 4n2 6n2for all n>1.

    • Therefore, by definition, 2n2 + 4n is O(n2).

  • Show that n3+n2logn+n is O(n3)

    • Since n n3 and log n  n for all n>1,we have n2 log n  n3, andn3 + n2log n + n 3n3for all n>1.

    • Therefore, by definition, n3+n2logn+n is O(n3).


Exercise l.jpg
Exercise

  • Determine the order of magnitude of the following functions.

  • n3 + 3n2 + 3

  • 4n2 log n + n3 + 5n2 + n

  • 2n2 + n2 log n

  • 6n2 + 2n

O(n3)

O(n3)

O(n2 log n)

O(2n)


Exercise 2 l.jpg
Exercise (2)

  • n3 = n3 n

  • 3n2  3n3 n1

  • 3  3n3n1

  •  n3+3n2+3  7n3 n1

  • Prove the order of magnitude

  • n3 + 3n2 + 3

    • n3 = n3 n

    • 3n2  n3 n3

    • 3  n3n2

       n3 + 3n2 + 3  3n3 n3

  • 4n2 log n + n3 + 5n2 + n

    • 4n2 log n  4n3 n1

    • n3 = n3 n

    • 5n2  n3 n5

    • nn3 n1

       4n2 log n + n3 + 5n2 + n  7n3 n5

  • 4n2 log n  4n3 n1

  • n3 = n3 n

  • 5n2  5n3 n1

  • nn3 n1

     4n2log n+n3+5n2+n  11n3 n1


Exercise 2 cont d l.jpg
Exercise (2) cont’d

  • 2n2  2n2log n n2

  • n2 log n = n2 log n n

     2n2+n2log n  3n2log n n2

  • Prove the order of magnitude

  • 2n2 + n2 log n

    • 2n2  n2 log n n4

    • n2 log n = n2 log n n

       2n2 + n2 log n  2n2 log n n4

  • 6n2 + 2n

    • 6n2  6*2n n1

    • 2n = 2n n

       6n2 + 2n  7*2n n1


More exercise l.jpg
More Exercise

  • Are the followings correct?

  • n2log n + n3 + 3n2 + 3 O(n2)?

  • n + 1000 O(n)?

  • 6n20 + 2n O(n20)?

  • n3 + 5n2 logn + n O(n2 log n) ?

O(n3)

YES

O(2n)

O(n3)


Some algorithms we learnt l.jpg
Some algorithms we learnt

input n

sum = 0

for i = 1 to n do

begin

sum = sum + i

end

output sum

  • Computing sum of the first n numbers

O(n)

O(1)

input n

sum = n*(n+1)/2

output sum

O(?)

O(?)

Finding the min value among n numbers

min = 0

for i = 1 to n-1 do

if (a[i] < a[min]) then

min = i

output a[min]

O(n)

O(?)


More exercise42 l.jpg
More exercise

  • What is the time complexity of the following pseudo code?

for i = 1 to 2n do

for j = 1 to n do

x = x + 1

O(n2)

O(?)

The outer loop iterates for 2n times.

The inner loop iterates for n times for each i.

Total: 2n * n = 2n2.


Learning outcomes43 l.jpg
Learning outcomes

  • Able to carry out simple asymptotic analysis of algorithms

  • See some examples of polynomial time and exponential time algorithms

    • Able to apply searching/sorting algorithms and derive their time complexities



Searching l.jpg
Searching

  • Input: a sequence of n numbers a0, a1, …, an-1; and a number X

  • Output: determine whether X is in the sequence or not

  • Algorithm (Linear search):

    • Starting from i=0, compare X with ai one by one as long as i < n.

    • Stop and report "Found!" when X = ai .

    • Repeat and report "Not Found!" when i >= n.


Linear search l.jpg
Linear Search

To find 7

  • 12 34 2 9 7 5 six numbers7number X

  • 12 34 2 9 7 57

  • 12 34 2 9 7 57

  • 12 34 2 9 7 57

  • 12 34 2 9 7 57found!


Linear search 2 l.jpg
Linear Search (2)

To find 10

  • 12 34 2 9 7 510

  • 12 34 2 9 7 510

  • 12 34 2 9 7 510

  • 12 34 2 9 7 510

  • 12 34 2 9 7 510

  • 12 34 2 9 7 510not found!


Linear search 3 l.jpg
Linear Search (3)

  • i = 0

  • while i < n do

  • begin

  • if X == a[i] then

  • report "Found!" and stop

  • else

  • i = i+1

  • end

  • report "Not Found!"


Time complexity l.jpg
Time Complexity

i = 0

while i < n do

begin

if X == a[i] then

report "Found!" & stop

else

i = i+1

end

report "Not Found!"

  • Important operation of searching: comparison

  • How many comparisons this algorithm requires?

Best case: X is the 1st no., 1 comparison, O(1)

Worst case: X is the last no. OR X is not found, n comparisons, O(n)


Improve time complexity l.jpg
Improve Time Complexity?

If the numbers are pre-sorted, then we can improve the time complexity of searching by binary search.


Binary search l.jpg
Binary Search

  • more efficient way of searching when the sequence of numbers is pre-sorted

  • Input: a sequence of n sorted numbers a0, a1, …, an-1 in ascending order and a number X

  • Idea of algorithm:

    • compare X with number in the middle

    • then focus on only the first half or the second half (depend on whether X is smaller or greater than the middle number)

    • reduce the amount of numbers to be searched by half


Binary search 2 l.jpg
Binary Search (2)

To find 24

  • 3 7 11 12 15 19 24 33 41 55 10 nos24X

  • 19 24 33 41 5524

  • 19 2424

  • 2424found!


Binary search 3 l.jpg
Binary Search (3)

To find 30

  • 3 7 11 12 15 19 24 33 41 55 10 nos30X

  • 19 24 33 41 5530

  • 19 2430

  • 2430not found!


Binary search 4 l.jpg
Binary Search (4)

  is the floor function,

truncate the decimal part

  • first=0, last=n-1

  • while (first <= last) dobegin

  • mid = (first+last)/2

  • if (X == a[mid]) report "Found!" & stop

  • else if (X < a[mid])last = mid-1

  • elsefirst = mid+1end

  • report "Not Found!"


Time complexity55 l.jpg
Time Complexity

Best case:X is the number in the middle=> 1 comparison, O(1)-time

Worst case:at most log2n+1 comparisons, O(log n)-time

Why? Every comparison reduces the amount of numbers by at least half

E.g., 16 => 8 => 4 => 2 => 1

first=0, last=n-1

while (first <= last) dobegin

mid = (first+last)/2

if (X == a[mid])

report "Found!" & stop

else if (X < a[mid]) last = mid-1

else

first = mid+1end

report "Not Found!"


Binary search vs linear search l.jpg
Binary search vs Linear search

  • Time complexity of linear search is O(n)

  • Time complexity of binary search is O(log n)

  • Therefore, binary search is more efficient than linear search


Search for a pattern l.jpg
Search for a pattern

  • We’ve seen how to search a number over a sequence of numbers

  • What about searching a pattern of characters over some text?

Example

text:

N O B O D Y _ N O T I C E _ H I M

pattern:

N O T

substring:

N O B O D Y _ N O T I C E _ H I M


String matching l.jpg
String Matching

  • Given a string of n characters called the text and a string of m characters (mn) called the pattern.

  • We want to determine if the text contains a substring matching the pattern.

Example

text:

N O B O D Y _ N O T I C E _ H I M

pattern:

N O T

substring:

N O B O D Y _ N O T I C E _ H I M


Example l.jpg
Example

T[ ]:

N O B O D Y _ N O T I C E _ H I M

N OT

P[ ]:

NO T

NO T

bolded: match

underlined: not match

un-bolded: not considered

NO T

NO T

NO T

NO T

N O T


The algorithm l.jpg
The algorithm

  • The algorithm scans over the text position by position.

  • For each position i, it checks whether the pattern P[0..m-1] appears in T[i..i+m-1]

  • If the pattern exists, then report found

  • Else continue with the next position i+1

  • If repeating until the end without success, report not found


Match pattern with t i i m 1 l.jpg
Match pattern with T[i..i+m-1]

  • j = 0

  • while (j < m && P[j]==T[i+j]) do

  • j = j + 1

  • if (j == m) then

  • report "found!" & stop

T[i] T[i+1] T[i+2] T[i+3] … T[i+m-1]

P[0] P[1] P[2] P[3] … P[m-1]


Match for all positions l.jpg
Match for all positions

  • for i = 0 to n-m do

  • begin

  • // check if P[0..m-1] match with T[i..i+m-1]

  • end

  • report "Not found!"


Match for all positions63 l.jpg
Match for all positions

  • for i = 0 to n-m do

  • begin

  • j = 0

  • while (j < m && P[j]==T[i+j]) do

  • j = j + 1

  • if (j == m) then

  • report "found!" & stop

  • end

  • report "Not found!"


Time complexity64 l.jpg
Time Complexity

How many comparisons this algorithm requires?

Best case:pattern appears in the beginning of the text, O(m)-time

Worst case:pattern appears at the end of the text OR pattern does not exist, O(nm)-time

for i = 0 to n-m do

begin

j = 0

while j < m & P[j]==T[i+j] do

j = j + 1

if j == m then

report "found!" & stop

end

report "Not found!"



Sorting l.jpg
Sorting

  • Input: a sequence of n numbers a0, a1, …, an-1

  • Output: arrange the n numbers into ascending order, i.e., from smallest to largest

  • Example: If the input contains 5 numbers 132, 56, 43, 200, 10, then the output should be 10, 43, 56, 132, 200

There are many sorting algorithms:insertion sort, selection sort, bubble sort, merge sort, quick sort


Selection sort l.jpg
Selection Sort

  • find minimum key from the input sequence

  • delete it from input sequence

  • append it to resulting sequence

  • repeat until nothing left in input sequence


Selection sort example l.jpg
Selection Sort - Example

  • sort (34, 10, 64, 51, 32, 21) in ascending order

    Sorted part Unsorted part Swapped

    34 10 64 51 32 21 10, 34

    10 34 64 51 32 21 21, 34

    10 21 64 51 32 34 32, 64

    10 21 32 51 64 34 51, 34

    10 21 32 34 64 51 51, 64

    10 21 32 34 51 64 --

    10 21 32 34 51 64


Selection sort algorithm l.jpg
Selection Sort Algorithm

  • for i = 0 to n-2 do

  • begin

  • // find the index of the minimum number

  • // in the range a[i] to a[n-1]

  • swap a[i] and a[min]

  • end


Selection sort algorithm70 l.jpg
Selection Sort Algorithm

how to swap two entries of an array?

using a temp variable?

not using a temp variable?

  • for i = 0 to n-2 do

  • begin

  • min = i

  • for j = i+1 to n-1 do

  • if a[j] < a[min] then

  • min = j

  • swap a[i] and a[min]

  • end


Algorithm analysis l.jpg
Algorithm Analysis

for i = 0 to n-2 do

begin

min = i

for j = i+1 to n-1 do

if a[j] < a[min] then

min = j

swap a[i] and a[min]

end

  • The algorithm consists of a nested for-loop.

  • For each iteration of the outer i-loop,there is an inner j-loop.

Total number of comparisons= (n-1) + (n-2) + … + 1= n(n-1)/2

O(n2)-time


Bubble sort l.jpg
Bubble Sort

  • starting from the last element, swap adjacent items if they are not in ascending order

  • when first item is reached, the first item is the smallest

  • repeat the above steps for the remaining items to find the second smallest item, and so on


Bubble sort example l.jpg
Bubble Sort - Example

  • (34 10 64 51 32 21)round

  • 34 10 64 51 32 21

  • 1 34 10 64 51 21 32

  • 34 10 64 21 51 32

  • 3410 21 64 51 32don’t need to swap

  • 34 10 21 64 51 32

  • 10 34 21 64 51 32

  • 2 10 34 21 64 32 51

  • 10 3421 32 64 51 don’t need to swap

  • 1034 21 32 64 51

  • 10 21 34 32 64 51

underlined: being considered

italic: sorted


Bubble sort example 2 l.jpg
Bubble Sort - Example (2)

  • round

  • 10 21 34 32 64 51

  • 3 10 21 3432 51 64don’t need to swap

  • 10 2134 32 51 64

  • 10 21323451 64don’t need to swap

  • 4 10 213234 5164don’t need to swap

  • 10 21323451 64don’t need to swap

  • 5 10 21 32 34 5164

underlined: being considered

italic: sorted


Bubble sort algorithm l.jpg
Bubble Sort Algorithm

the smallest will be moved to a[i]

  • for i = 0 to n-2 do

  • for j = n-1 downto i+1 do

  • if (a[j] < a[j-1])

  • swap a[j] & a[j-1]

start from a[n-1], check up to a[i+1]

34 10 64 51 32 21

i = 0

j = 5

j = 4

j = 3

j = 2

j = 1

i = 1

j = 5

j = 4

j = 3

j = 2


Algorithm analysis76 l.jpg
Algorithm Analysis

  • The algorithm consists of a nested for-loop.

for i = 0 to n-2 do

for j = n-1 downto i+1 do

if (a[j] < a[j-1])

swap a[j] & a[j-1]

Total number of comparisons= (n-1) + (n-2) + … + 1= n(n-1)/2

O(n2)-time


Sorting77 l.jpg
Sorting

  • Input: a sequence of n numbers a0, a1, …, an-1

  • Output: arrange the n numbers into ascending order, i.e., from smallest to largest

We have learnt these sorting algorithms:selection sort, bubble sort

Next: insertion sort (optional, self-study)


Insertion sort optional self study l.jpg
Insertion Sort (optional, self-study)

  • look at elements one by one

  • build up sorted list by inserting the element at the correct location

optional


Example79 l.jpg
Example

sort (34, 8, 64, 51, 32, 21) in ascending order

Sorted part Unsorted part int moved

34 8 64 51 32 21

34 8 64 51 32 21 -

8 34 64 51 32 21 34

8 34 64 51 32 21 -

8 34 51 64 32 21 64

8 32 34 51 64 21 34, 51, 64

8 21 32 34 51 64 32, 34, 51, 64

optional


Insertion sort algorithm l.jpg
Insertion Sort Algorithm

  • for i = 1 to n-1 do

  • begin key = a[i]

  • pos = 0while (a[pos] < key) && (pos < i) do pos = pos + 1

  • shift a[pos], …, a[i-1] to the right

  • a[pos] = key

  • end

using linear search to find the correct position for key

i.e., move a[i-1] to a[i], a[i-2] to a[i-1], …, a[pos] to a[pos+1]

finally, place key (the original a[i]) in a[pos]

optional


Algorithm analysis81 l.jpg
Algorithm Analysis

for i = 1 to n-1 do

begin key = a[i]

pos = 0while (a[pos] < key) && (pos < i) do pos = pos + 1

shift a[pos], …, a[i-1] to the right

a[pos] = key

end

  • Worst case input

    • input is sorted in descending order

  • Then, for a[i]

    • finding the position takes i comparisons

total number of comparisons= 1 + 2 + … + n-1= (n-1)n/2

O(n2)-time

optional


Selection bubble insertion sort l.jpg
Selection, Bubble, Insertion Sort

  • All three algorithms have time complexity O(n2) in the worst case.

  • Are there any more efficient sorting algorithms? YES, we will learn them later.

  • What is the time complexity of the fastest comparison-based sorting algorithm?O(n log n)



Traveling salesman problem l.jpg
Traveling Salesman Problem Problem, Knapsack Problem …

  • Input: There are n cities.

  • Output: Find the shortest tour from a particular city that visit each city exactly once before returning to the city where it started.

This is known as Hamiltonian circuit


Example85 l.jpg
Example Problem, Knapsack Problem …

2

a

b

To find a Hamiltonian circuit from a to a

7

8

3

5

c

d

1

TourLength

a -> b -> c -> d -> a 2 + 8 + 1 + 7 = 18

a -> b -> d -> c -> a 2 + 3 + 1 + 5 = 11

a -> c -> b -> d -> a 5 + 8 + 3 + 7 = 23

a -> c -> d -> b -> a 5 + 1 + 3 + 2 = 11

a -> d -> b -> c -> a 7 + 3 + 8 + 5 = 23

a -> d -> c -> b -> a 7 + 1 + 8 + 2 = 18


Idea and analysis l.jpg
Idea and Analysis Problem, Knapsack Problem …

  • A Hamiltonian circuit can be represented by a sequence of n+1 cities v1, v2, …, vn, v1, where the first and the last are the same, and all the others are distinct.

  • Exhaustive search approach: Find all tours in this form, compute the tour length and find the shortest among them.

How many possible tours to consider?

(n-1)! = (n-1)(n-2)…1

N.B.: (n-1)! is exponential in terms of n

[ refer to batch 1 notes, slides #64,65 ]


Knapsack problem l.jpg
Knapsack Problem Problem, Knapsack Problem …

  • Input: Given n items with weights w1, w2, …, wn and values v1, v2, …, vn, and a knapsack with capacity W.

  • Output: Find the most valuable subset of items that can fit into the knapsack.

  • Application: A transport plane is to deliver the most valuable set of items to a remote location without exceeding its capacity.


Example88 l.jpg

total total Problem, Knapsack Problem … subsetweightvalue

 0 0

{1} 7 42

{2} 3 12

{3} 4 40

{4} 5 25

{1,2} 10 54

{1,3} 11 N/A

{1,4} 12 N/A

total totalsubsetweightvalue

{2,3} 7 52

{2,4} 8 37

{3,4} 9 65

{1,2,3} 14 N/A

{1,2,4} 15 N/A

{1,3,4} 16 N/A

{2,3,4} 12 N/A

{1,2,3,4} 19 N/A

Example

w = 7

v = 42

capacity = 10

w = 5

v = 25

w = 4

v = 40

w = 3

v = 12

item 1

item 2

item 3

item 4

knapsack


Idea and analysis89 l.jpg
Idea and Analysis Problem, Knapsack Problem …

  • Exhaustive search approach: Try every subset of the set of n given items, compute total weight of each subset and compute total value of those subsets that do NOT exceed knapsack's capacity.

How many subsets to consider?

2n, why?


Exercises 1 l.jpg
Exercises (1) Problem, Knapsack Problem …

  • Suppose you have forgotten a password with 5 characters. You only remember:

    • the 5 characters are all distinct

    • the 5 characters are B, D, M, P, Y

  • If you want to try all possible combinations, how many of them in total?

  • What if the 5 characters can be any of the 26 upper case alphabet?

5! = 120

26*25*24*23*22

or 26!/21!


Exercises 2 l.jpg
Exercises (2) Problem, Knapsack Problem …

  • Suppose the password also contains 2 digits, i.e., 7 characters in total

    • all characters are all distinct

    • the 5 alphabets are B, D, M, P, Y

    • the digit is either 0 or 1

  • How many combinations are there?

7! = 5040


Exercises 3 l.jpg
Exercises (3) Problem, Knapsack Problem …

  • What if the password is in the form adaaada?

    • a means alphabet, d means digit

    • all characters are all distinct

    • the 5 alphabets are B, D, M, P, Y

    • the digit is either 0 or 1

  • How many combinations are there?

5! * 2! = 240


Learning outcomes93 l.jpg
Learning outcomes Problem, Knapsack Problem …

  • Able to carry out simple asymptotic analysis of algorithms

  • See some examples of polynomial time and exponential time algorithms

    • Able to apply searching/sorting algorithms and derive their time complexities


ad