comp108 algorithmic foundations algorithm efficiency searching sorting
Download
Skip this Video
Download Presentation
COMP108 Algorithmic Foundations Algorithm efficiency + Searching/Sorting

Loading in 2 Seconds...

play fullscreen
1 / 93

comp108 algorithmic foundations algorithm efficiency searching - PowerPoint PPT Presentation


  • 281 Views
  • Uploaded on

COMP108 Algorithmic Foundations Algorithm efficiency + Searching/Sorting. Prudence Wong http://www.csc.liv.ac.uk/~pwong/teaching/comp108. Learning outcomes. Able to carry out simple asymptotic analysis of algorithms See some examples of polynomial time and exponential time algorithms

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'comp108 algorithmic foundations algorithm efficiency searching' - Gabriel


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
comp108 algorithmic foundations algorithm efficiency searching sorting

COMP108Algorithmic FoundationsAlgorithm efficiency + Searching/Sorting

Prudence Wong

http://www.csc.liv.ac.uk/~pwong/teaching/comp108

learning outcomes
Learning outcomes
  • Able to carry out simple asymptotic analysis of algorithms
  • See some examples of polynomial time and exponential time algorithms
    • Able to apply searching/sorting algorithms and derive their time complexities
time complexity analysis
Time Complexity Analysis
  • How fast is the algorithm?

Code the algorithm and run the program, then measure the running time

  • Depend on the speed of the computer
  • Waste time coding and testing if the algorithm is slow

Identify some important operations/steps and count how many times these operations/steps needed to executed

time complexity analysis4
Time Complexity Analysis
  • How to measure efficiency?

Number of operations usually expressed in terms of input size

  • If we doubled the input size, how much longer would the algorithm take?If we trebled the input size, how much longer would it take?
why efficiency matters
Why efficiency matters?
  • speed of computation by hardware has been improved
  • efficiency still matters
  • ambition for computer applications grow with computer power
  • demand a great increase in speed of computation
amount of data handled matches speed increase
Amount of data handled matches speed increase?
  • When computation speed vastly increased, can we handle much more data?
  • Suppose
    • an algorithm takes n2comparisons to sortn numbers
    • we need 1 sec to sort 5 numbers (25 comparisons)
    • computing speed increases by factor of100
  • Using 1 sec, we can now perform100x25comparisons,i.e., to sort50 numbers
  • With 100 times speedup, only sort 10 times more numbers!
time complexity analysis7
Time Complexity Analysis

input n

sum = 0

for i = 1 to n do

begin

sum = sum + i

end

output sum

  • Important operation of summation: addition
  • How many additions this algorithm requires?

We need n additions

(depend on the input size n)

space complexity analysis
Space Complexity Analysis

input n

sum = 0

for i = 1 to n do

begin

sum = sum + i

end

output sum

We need to store the number n, the variable sum, and the index i

=> needs 3 memory space

In this example, the memory space does not depend on the input size n

In other case, it may depend on the input size n

look for improvement
Look for improvement

input n

sum = n*(n+1)/2

output sum

Mathematical formula gives us an alternative to find the sum of first n numbers:

1 + 2 + ... + n = n(n+1)/2

We only need 3 operations:

1 addition, 1 multiplication, and 1 division

(no matter what the input size n is)

finding min among 3 numbers
Finding min among 3 numbers

Input: 3 numbers a, b, c

Output: the min value of these 3 numbers

Algorithm:

Input a, b, c

a  b ?

N

Y

a  c ?

Y

b  c ?

N

Y

N

Output a

Output b

Output c

Output c

finding min among 3 numbers12
Finding min among 3 numbers

Example 1

Input a, b, c

a=3, b=5, c=2

Y

N

a  b ?

a  c ?

Y

b  c ?

N

Y

N

Output a

Output b

Output c

Output c

2 is output

finding min among 3 numbers13
Finding min among 3 numbers

Example 2

Input a, b, c

a=6, b=5, c=7

Y

N

a  b ?

a  c ?

Y

b  c ?

N

Y

N

Output a

Output b

Output c

Output c

5 is output

time complexity analysis14
Time Complexity Analysis
  • Important operation: comparison
  • How many comparison this algorithm requires?

input a, b, c

if (a  b) then

if (a  c) then

output a

else

output c

else

if (b  c) then

output b

else

output c

It takes 2 comparisons

finding min among n numbers
Finding min among n numbers

Input a0, a1, ...

Any systematic approach?

a0  a1 ?

N

Y

a0  a2 ?

Y

a1  a2 ?

Y

N

N

a0  a3 ?

a1  a3 ?

a2  a3 ?

a2  a3 ?

finding min among n numbers16
Finding min among n numbers

Input a[0], a[1], ..., a[n-1]

min = 0

i = 1

i = i+1

i < n?

N

Y

Output a[min]

a[i] < a[min]?

Y

min = i

N

finding min among n numbers17
Finding min among n numbers

input a[0], a[1], ..., a[n-1]

min = 0

i = 1

while (i <n) do

begin

if (a[i] < a[min]) then

min = i

i = i + 1

end

output a[min]

Example

a[]={50,30,40,20,10}

0

50

1

30

1

30

3

20

4

10

finding min among n numbers18
Finding min among n numbers

n-1

  • Time complexity: ?? comparisons

input a[0], a[1], ..., a[n-1]

min = 0

i = 1

while (i <n) do

begin

if (a[i] < a[min]) then

min = i

i = i + 1

end

output a[min]

finding min using for loop
Finding min using for-loop
  • Rewrite the above while-loop into a for-loop

input a[0], a[1], ..., a[n-1]

min = 0

for i = 1 to n-1 do

begin

if (a[i] < a[min]) then

min = i

end

output a[min]

which algorithm is the fastest
Which algorithm is the fastest?
  • Consider a problem that can be solved by 5 algorithms A1, A2, A3, A4, A5 using different number of operations (time complexity).

f1(n) = 50n + 20

f2(n) = 10 n log2 n + 100

f3(n) = n2 - 3n + 6

f4(n) = 2n2

f5(n) = 2n/8 - n/4 + 2

f5(n)

f3(n)

f1(n)

Depends on the size of the problem!

slide22

f1(n) = 50n + 20

f2(n) = 10 n log2n + 100

f3(n) = n2 - 3n + 6

f4(n) = 2n2

f5(n) = 2n/8 - n/4 + 2

what do we observe
What do we observe?
  • There is huge difference between
    • functions involving powers ofn (e.g., n, n log n, n2, called polynomial functions) and
    • functions involving powering by n (e.g., 2n, called exponential functions)
  • Among polynomial functions, those with same order of power are more comparable
    • e.g., f3(n) = n2 - 3n + 6 and f4(n) = 2n2
hierarchy of functions
Hierarchy of functions
  • We can define a hierarchy of functions each having a greater order of magnitude than its predecessor:

1

log n

n n2 n3 ... nk ...

2n

constant

logarithmic

polynomial

exponential

  • We can further refine the hierarchy by inserting n log n between n and n2,n2 log n between n2 and n3, and so on.
hierarchy of functions 2
Hierarchy of functions (2)

1

log n

n n2 n3 ... nk ...

2n

  • Important: as we move from left to right, successive functions have greater order of magnitude than the previous ones.
  • As n increases, the values of the later functions increase more rapidly than the values of the earlier ones.
  • Relative growth rates increase

constant

logarithmic

polynomial

exponential

hierarchy of functions 3
Hierarchy of functions (3)

1

log n

n n2 n3 ... nk ...

2n

  • Now, when we have a function, we can assign the function to some function in the hierarchy:
    • For example, f(n) = 2n3 + 5n2 + 4n + 7The term with the highest power is 2n3.The growth rate of f(n) is dominated by n3.
  • This concept is captured by Big-O notation

constant

logarithmic

polynomial

exponential

big o notation
Big-O notation
  • f(n) = O(g(n)) [read as f(n) is of order g(n)]
  • Roughly speaking, this means f(n) is at most a constant times g(n) for all large n
  • Examples
    • 2n3 = O(n3)
    • 3n2 = O(n2)
    • 2n log n = O(n log n)
    • n3 + n2 = O(n3)
  • function on L.H.S and function on R.H.S are said to have the same order of magnitude
big o notation formal definition
Big-O notation - formal definition
  • f(n) = O(g(n))
  • There exists a constant c and no such that f(n) cg(n) for all n > no
    • cnon>no then f(n)  cg(n)
f n o g n graphical illustration
f(n) = O(g(n) – Graphical illustration

constant c & no such thatn>no, f(n)  c g(n)

c g(n)

f(n)

n0

example n 60 o n
Example – n+60 = O(n)

2n

n + 60

n

c=2, n0=60

n0

constant c & no such that n>no, f(n)  c g(n)

example 2nlog 2 n 100 n 1000 o n log n
Example – 2nlog2n+100n+1000 = O(n log n)

3 n log n

c=3, n0=600

f(n)

n log n

n0

constant c & no such that n>no, f(n)  c g(n)

f n o g n
f(n) = O(g(n))?

c g(n)

f(n)

value of f(n) for n<n0does NOT matter

n0

constant c & no such that n>no, f(n)  c g(n)

which one is the fastest
Which one is the fastest?
  • Usually we are only interested in the asymptotic time complexity, i.e., when n is large
  • O(log n) < O(n) < O(n) < O(n log n) < O(n2) < O(2n)
proof of order of magnitude
Proof of order of magnitude
  • Show that 2n2 + 4n is O(n2)
    • Since n  n2 for all n>1,we have 2n2 + 4n  2n2 + 4n2 6n2for all n>1.
    • Therefore, by definition, 2n2 + 4n is O(n2).
  • Show that n3+n2logn+n is O(n3)
    • Since n n3 and log n  n for all n>1,we have n2 log n  n3, andn3 + n2log n + n 3n3for all n>1.
    • Therefore, by definition, n3+n2logn+n is O(n3).
exercise
Exercise
  • Determine the order of magnitude of the following functions.
  • n3 + 3n2 + 3
  • 4n2 log n + n3 + 5n2 + n
  • 2n2 + n2 log n
  • 6n2 + 2n

O(n3)

O(n3)

O(n2 log n)

O(2n)

exercise 2
Exercise (2)
  • n3 = n3 n
  • 3n2  3n3 n1
  • 3  3n3n1
  •  n3+3n2+3  7n3 n1
  • Prove the order of magnitude
  • n3 + 3n2 + 3
    • n3 = n3 n
    • 3n2  n3 n3
    • 3  n3n2

 n3 + 3n2 + 3  3n3 n3

  • 4n2 log n + n3 + 5n2 + n
    • 4n2 log n  4n3 n1
    • n3 = n3 n
    • 5n2  n3 n5
    • nn3 n1

 4n2 log n + n3 + 5n2 + n  7n3 n5

  • 4n2 log n  4n3 n1
  • n3 = n3 n
  • 5n2  5n3 n1
  • nn3 n1

 4n2log n+n3+5n2+n  11n3 n1

exercise 2 cont d
Exercise (2) cont’d
  • 2n2  2n2log n n2
  • n2 log n = n2 log n n

 2n2+n2log n  3n2log n n2

  • Prove the order of magnitude
  • 2n2 + n2 log n
    • 2n2  n2 log n n4
    • n2 log n = n2 log n n

 2n2 + n2 log n  2n2 log n n4

  • 6n2 + 2n
    • 6n2  6*2n n1
    • 2n = 2n n

 6n2 + 2n  7*2n n1

more exercise
More Exercise
  • Are the followings correct?
  • n2log n + n3 + 3n2 + 3 O(n2)?
  • n + 1000 O(n)?
  • 6n20 + 2n O(n20)?
  • n3 + 5n2 logn + n O(n2 log n) ?

O(n3)

YES

O(2n)

O(n3)

some algorithms we learnt
Some algorithms we learnt

input n

sum = 0

for i = 1 to n do

begin

sum = sum + i

end

output sum

  • Computing sum of the first n numbers

O(n)

O(1)

input n

sum = n*(n+1)/2

output sum

O(?)

O(?)

Finding the min value among n numbers

min = 0

for i = 1 to n-1 do

if (a[i] < a[min]) then

min = i

output a[min]

O(n)

O(?)

more exercise42
More exercise
  • What is the time complexity of the following pseudo code?

for i = 1 to 2n do

for j = 1 to n do

x = x + 1

O(n2)

O(?)

The outer loop iterates for 2n times.

The inner loop iterates for n times for each i.

Total: 2n * n = 2n2.

learning outcomes43
Learning outcomes
  • Able to carry out simple asymptotic analysis of algorithms
  • See some examples of polynomial time and exponential time algorithms
    • Able to apply searching/sorting algorithms and derive their time complexities
searching
Searching
  • Input: a sequence of n numbers a0, a1, …, an-1; and a number X
  • Output: determine whether X is in the sequence or not
  • Algorithm (Linear search):
    • Starting from i=0, compare X with ai one by one as long as i < n.
    • Stop and report "Found!" when X = ai .
    • Repeat and report "Not Found!" when i >= n.
linear search
Linear Search

To find 7

  • 12 34 2 9 7 5 six numbers7number X
  • 12 34 2 9 7 57
  • 12 34 2 9 7 57
  • 12 34 2 9 7 57
  • 12 34 2 9 7 57found!
linear search 2
Linear Search (2)

To find 10

  • 12 34 2 9 7 510
  • 12 34 2 9 7 510
  • 12 34 2 9 7 510
  • 12 34 2 9 7 510
  • 12 34 2 9 7 510
  • 12 34 2 9 7 510not found!
linear search 3
Linear Search (3)
  • i = 0
  • while i < n do
  • begin
  • if X == a[i] then
  • report "Found!" and stop
  • else
  • i = i+1
  • end
  • report "Not Found!"
time complexity
Time Complexity

i = 0

while i < n do

begin

if X == a[i] then

report "Found!" & stop

else

i = i+1

end

report "Not Found!"

  • Important operation of searching: comparison
  • How many comparisons this algorithm requires?

Best case: X is the 1st no., 1 comparison, O(1)

Worst case: X is the last no. OR X is not found, n comparisons, O(n)

improve time complexity
Improve Time Complexity?

If the numbers are pre-sorted, then we can improve the time complexity of searching by binary search.

binary search
Binary Search
  • more efficient way of searching when the sequence of numbers is pre-sorted
  • Input: a sequence of n sorted numbers a0, a1, …, an-1 in ascending order and a number X
  • Idea of algorithm:
    • compare X with number in the middle
    • then focus on only the first half or the second half (depend on whether X is smaller or greater than the middle number)
    • reduce the amount of numbers to be searched by half
binary search 2
Binary Search (2)

To find 24

  • 3 7 11 12 15 19 24 33 41 55 10 nos24X
  • 19 24 33 41 5524
  • 19 2424
  • 2424found!
binary search 3
Binary Search (3)

To find 30

  • 3 7 11 12 15 19 24 33 41 55 10 nos30X
  • 19 24 33 41 5530
  • 19 2430
  • 2430not found!
binary search 4
Binary Search (4)

  is the floor function,

truncate the decimal part

  • first=0, last=n-1
  • while (first <= last) dobegin
  • mid = (first+last)/2
  • if (X == a[mid]) report "Found!" & stop
  • else if (X < a[mid])last = mid-1
  • elsefirst = mid+1end
  • report "Not Found!"
time complexity55
Time Complexity

Best case:X is the number in the middle=> 1 comparison, O(1)-time

Worst case:at most log2n+1 comparisons, O(log n)-time

Why? Every comparison reduces the amount of numbers by at least half

E.g., 16 => 8 => 4 => 2 => 1

first=0, last=n-1

while (first <= last) dobegin

mid = (first+last)/2

if (X == a[mid])

report "Found!" & stop

else if (X < a[mid]) last = mid-1

else

first = mid+1end

report "Not Found!"

binary search vs linear search
Binary search vs Linear search
  • Time complexity of linear search is O(n)
  • Time complexity of binary search is O(log n)
  • Therefore, binary search is more efficient than linear search
search for a pattern
Search for a pattern
  • We’ve seen how to search a number over a sequence of numbers
  • What about searching a pattern of characters over some text?

Example

text:

N O B O D Y _ N O T I C E _ H I M

pattern:

N O T

substring:

N O B O D Y _ N O T I C E _ H I M

string matching
String Matching
  • Given a string of n characters called the text and a string of m characters (mn) called the pattern.
  • We want to determine if the text contains a substring matching the pattern.

Example

text:

N O B O D Y _ N O T I C E _ H I M

pattern:

N O T

substring:

N O B O D Y _ N O T I C E _ H I M

example
Example

T[ ]:

N O B O D Y _ N O T I C E _ H I M

N OT

P[ ]:

NO T

NO T

bolded: match

underlined: not match

un-bolded: not considered

NO T

NO T

NO T

NO T

N O T

the algorithm
The algorithm
  • The algorithm scans over the text position by position.
  • For each position i, it checks whether the pattern P[0..m-1] appears in T[i..i+m-1]
  • If the pattern exists, then report found
  • Else continue with the next position i+1
  • If repeating until the end without success, report not found
match pattern with t i i m 1
Match pattern with T[i..i+m-1]
  • j = 0
  • while (j < m && P[j]==T[i+j]) do
  • j = j + 1
  • if (j == m) then
  • report "found!" & stop

T[i] T[i+1] T[i+2] T[i+3] … T[i+m-1]

P[0] P[1] P[2] P[3] … P[m-1]

match for all positions
Match for all positions
  • for i = 0 to n-m do
  • begin
  • // check if P[0..m-1] match with T[i..i+m-1]
  • end
  • report "Not found!"
match for all positions63
Match for all positions
  • for i = 0 to n-m do
  • begin
  • j = 0
  • while (j < m && P[j]==T[i+j]) do
  • j = j + 1
  • if (j == m) then
  • report "found!" & stop
  • end
  • report "Not found!"
time complexity64
Time Complexity

How many comparisons this algorithm requires?

Best case:pattern appears in the beginning of the text, O(m)-time

Worst case:pattern appears at the end of the text OR pattern does not exist, O(nm)-time

for i = 0 to n-m do

begin

j = 0

while j < m & P[j]==T[i+j] do

j = j + 1

if j == m then

report "found!" & stop

end

report "Not found!"

sorting
Sorting
  • Input: a sequence of n numbers a0, a1, …, an-1
  • Output: arrange the n numbers into ascending order, i.e., from smallest to largest
  • Example: If the input contains 5 numbers 132, 56, 43, 200, 10, then the output should be 10, 43, 56, 132, 200

There are many sorting algorithms:insertion sort, selection sort, bubble sort, merge sort, quick sort

selection sort
Selection Sort
  • find minimum key from the input sequence
  • delete it from input sequence
  • append it to resulting sequence
  • repeat until nothing left in input sequence
selection sort example
Selection Sort - Example
  • sort (34, 10, 64, 51, 32, 21) in ascending order

Sorted part Unsorted part Swapped

34 10 64 51 32 21 10, 34

10 34 64 51 32 21 21, 34

10 21 64 51 32 34 32, 64

10 21 32 51 64 34 51, 34

10 21 32 34 64 51 51, 64

10 21 32 34 51 64 --

10 21 32 34 51 64

selection sort algorithm
Selection Sort Algorithm
  • for i = 0 to n-2 do
  • begin
  • // find the index of the minimum number
  • // in the range a[i] to a[n-1]
  • swap a[i] and a[min]
  • end
selection sort algorithm70
Selection Sort Algorithm

how to swap two entries of an array?

using a temp variable?

not using a temp variable?

  • for i = 0 to n-2 do
  • begin
  • min = i
  • for j = i+1 to n-1 do
  • if a[j] < a[min] then
  • min = j
  • swap a[i] and a[min]
  • end
algorithm analysis
Algorithm Analysis

for i = 0 to n-2 do

begin

min = i

for j = i+1 to n-1 do

if a[j] < a[min] then

min = j

swap a[i] and a[min]

end

  • The algorithm consists of a nested for-loop.
  • For each iteration of the outer i-loop,there is an inner j-loop.

Total number of comparisons= (n-1) + (n-2) + … + 1= n(n-1)/2

O(n2)-time

bubble sort
Bubble Sort
  • starting from the last element, swap adjacent items if they are not in ascending order
  • when first item is reached, the first item is the smallest
  • repeat the above steps for the remaining items to find the second smallest item, and so on
bubble sort example
Bubble Sort - Example
  • (34 10 64 51 32 21)round
  • 34 10 64 51 32 21
  • 1 34 10 64 51 21 32
  • 34 10 64 21 51 32
  • 3410 21 64 51 32don’t need to swap
  • 34 10 21 64 51 32
  • 10 34 21 64 51 32
  • 2 10 34 21 64 32 51
  • 10 3421 32 64 51 don’t need to swap
  • 1034 21 32 64 51
  • 10 21 34 32 64 51

underlined: being considered

italic: sorted

bubble sort example 2
Bubble Sort - Example (2)
  • round
  • 10 21 34 32 64 51
  • 3 10 21 3432 51 64don’t need to swap
  • 10 2134 32 51 64
  • 10 21323451 64don’t need to swap
  • 4 10 213234 5164don’t need to swap
  • 10 21323451 64don’t need to swap
  • 5 10 21 32 34 5164

underlined: being considered

italic: sorted

bubble sort algorithm
Bubble Sort Algorithm

the smallest will be moved to a[i]

  • for i = 0 to n-2 do
  • for j = n-1 downto i+1 do
  • if (a[j] < a[j-1])
  • swap a[j] & a[j-1]

start from a[n-1], check up to a[i+1]

34 10 64 51 32 21

i = 0

j = 5

j = 4

j = 3

j = 2

j = 1

i = 1

j = 5

j = 4

j = 3

j = 2

algorithm analysis76
Algorithm Analysis
  • The algorithm consists of a nested for-loop.

for i = 0 to n-2 do

for j = n-1 downto i+1 do

if (a[j] < a[j-1])

swap a[j] & a[j-1]

Total number of comparisons= (n-1) + (n-2) + … + 1= n(n-1)/2

O(n2)-time

sorting77
Sorting
  • Input: a sequence of n numbers a0, a1, …, an-1
  • Output: arrange the n numbers into ascending order, i.e., from smallest to largest

We have learnt these sorting algorithms:selection sort, bubble sort

Next: insertion sort (optional, self-study)

insertion sort optional self study
Insertion Sort (optional, self-study)
  • look at elements one by one
  • build up sorted list by inserting the element at the correct location

optional

example79
Example

sort (34, 8, 64, 51, 32, 21) in ascending order

Sorted part Unsorted part int moved

34 8 64 51 32 21

34 8 64 51 32 21 -

8 34 64 51 32 21 34

8 34 64 51 32 21 -

8 34 51 64 32 21 64

8 32 34 51 64 21 34, 51, 64

8 21 32 34 51 64 32, 34, 51, 64

optional

insertion sort algorithm
Insertion Sort Algorithm
  • for i = 1 to n-1 do
  • begin key = a[i]
  • pos = 0while (a[pos] < key) && (pos < i) do pos = pos + 1
  • shift a[pos], …, a[i-1] to the right
  • a[pos] = key
  • end

using linear search to find the correct position for key

i.e., move a[i-1] to a[i], a[i-2] to a[i-1], …, a[pos] to a[pos+1]

finally, place key (the original a[i]) in a[pos]

optional

algorithm analysis81
Algorithm Analysis

for i = 1 to n-1 do

begin key = a[i]

pos = 0while (a[pos] < key) && (pos < i) do pos = pos + 1

shift a[pos], …, a[i-1] to the right

a[pos] = key

end

  • Worst case input
    • input is sorted in descending order
  • Then, for a[i]
    • finding the position takes i comparisons

total number of comparisons= 1 + 2 + … + n-1= (n-1)n/2

O(n2)-time

optional

selection bubble insertion sort
Selection, Bubble, Insertion Sort
  • All three algorithms have time complexity O(n2) in the worst case.
  • Are there any more efficient sorting algorithms? YES, we will learn them later.
  • What is the time complexity of the fastest comparison-based sorting algorithm?O(n log n)
traveling salesman problem
Traveling Salesman Problem
  • Input: There are n cities.
  • Output: Find the shortest tour from a particular city that visit each city exactly once before returning to the city where it started.

This is known as Hamiltonian circuit

example85
Example

2

a

b

To find a Hamiltonian circuit from a to a

7

8

3

5

c

d

1

TourLength

a -> b -> c -> d -> a 2 + 8 + 1 + 7 = 18

a -> b -> d -> c -> a 2 + 3 + 1 + 5 = 11

a -> c -> b -> d -> a 5 + 8 + 3 + 7 = 23

a -> c -> d -> b -> a 5 + 1 + 3 + 2 = 11

a -> d -> b -> c -> a 7 + 3 + 8 + 5 = 23

a -> d -> c -> b -> a 7 + 1 + 8 + 2 = 18

idea and analysis
Idea and Analysis
  • A Hamiltonian circuit can be represented by a sequence of n+1 cities v1, v2, …, vn, v1, where the first and the last are the same, and all the others are distinct.
  • Exhaustive search approach: Find all tours in this form, compute the tour length and find the shortest among them.

How many possible tours to consider?

(n-1)! = (n-1)(n-2)…1

N.B.: (n-1)! is exponential in terms of n

[ refer to batch 1 notes, slides #64,65 ]

knapsack problem
Knapsack Problem
  • Input: Given n items with weights w1, w2, …, wn and values v1, v2, …, vn, and a knapsack with capacity W.
  • Output: Find the most valuable subset of items that can fit into the knapsack.
  • Application: A transport plane is to deliver the most valuable set of items to a remote location without exceeding its capacity.
example88

total totalsubsetweightvalue

 0 0

{1} 7 42

{2} 3 12

{3} 4 40

{4} 5 25

{1,2} 10 54

{1,3} 11 N/A

{1,4} 12 N/A

total totalsubsetweightvalue

{2,3} 7 52

{2,4} 8 37

{3,4} 9 65

{1,2,3} 14 N/A

{1,2,4} 15 N/A

{1,3,4} 16 N/A

{2,3,4} 12 N/A

{1,2,3,4} 19 N/A

Example

w = 7

v = 42

capacity = 10

w = 5

v = 25

w = 4

v = 40

w = 3

v = 12

item 1

item 2

item 3

item 4

knapsack

idea and analysis89
Idea and Analysis
  • Exhaustive search approach: Try every subset of the set of n given items, compute total weight of each subset and compute total value of those subsets that do NOT exceed knapsack\'s capacity.

How many subsets to consider?

2n, why?

exercises 1
Exercises (1)
  • Suppose you have forgotten a password with 5 characters. You only remember:
    • the 5 characters are all distinct
    • the 5 characters are B, D, M, P, Y
  • If you want to try all possible combinations, how many of them in total?
  • What if the 5 characters can be any of the 26 upper case alphabet?

5! = 120

26*25*24*23*22

or 26!/21!

exercises 2
Exercises (2)
  • Suppose the password also contains 2 digits, i.e., 7 characters in total
    • all characters are all distinct
    • the 5 alphabets are B, D, M, P, Y
    • the digit is either 0 or 1
  • How many combinations are there?

7! = 5040

exercises 3
Exercises (3)
  • What if the password is in the form adaaada?
    • a means alphabet, d means digit
    • all characters are all distinct
    • the 5 alphabets are B, D, M, P, Y
    • the digit is either 0 or 1
  • How many combinations are there?

5! * 2! = 240

learning outcomes93
Learning outcomes
  • Able to carry out simple asymptotic analysis of algorithms
  • See some examples of polynomial time and exponential time algorithms
    • Able to apply searching/sorting algorithms and derive their time complexities
ad