Loading in 5 sec....

Introduction to AlgorithmPowerPoint Presentation

Introduction to Algorithm

- 104 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about 'Introduction to Algorithm' - brooklyn

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Introduction to Algorithm

Algorithm:

Well-defined computational procedure that

takes some value(s) (called input) and produces

some value(s) (called output).

Or a sequence of computational steps that

transforms the input into the output.

Example: sorting problem.

Input: a sequence of n number a1, a2, …,an

Output: a permutation (reordering) a1', a2', …,an'

such that a1' a2' … an '.

Problem instance: <31,41,59,26,41,58> <26,31,41,41,58,59>

Correctness: an algorithm is said to be correct if for every

input instance, it halts with correct output.

Algorithm Design and Analysis

- Data structure: the way to store and organize data.
- Techniques:
- different design and analysis techniques in this course.
- purpose is for you to design/analyze your own algorithms for new problems in the future.

- Hard problem:
- Most problems discussed are efficient (poly time)
- An interesting set of hard problems: NP-complete.

NP-complete problem

- Why interesting:
- Not known whether an efficient algorithm exists for them.
- If exist for one, then exist for all.
- A small change may cause big change.

- Why important:
- Arise surprisingly often in real world.
- Not waste time on trying to find efficient algorithm to get best solution, instead find approximate or near-optimal solution.

- Example: traveling-salesman problem.

Efficiency of Algorithms

- Algorithms for solving the same problem can differ dramatically in their efficiency.
- much more significant than the differences due to hardware and software.
- Comparison of two sorting algorithms (n=106 numbers):
- Insertion sort: c1n2
- Merge sort: c2n (lg n)
- Best programmer (c1=2), machine language, one billion/second computer.
- Bad programmer (c2=50), high-language, ten million/second computer.
- 2 (106)2 instructions/109 instructions per second = 2000 seconds.
- 50 (106 lg 106) instructions/107 instructions per second 100 seconds.
- Thus, merge sort on B is 20 times faster than insertion sort on A!
- If sorting ten million number, 2.3 days VS. 20 minutes.

Algorithms as technology

- Other modern technologies:
- Hardware with high clock rate, pipeline, ….
- Easy-to-use, GUI
- Object-oriented systems
- LAN and WAN

- Algorithms are no less important than these.
- Moreover, all these new technologies involve heavily algorithms.
- More large and complicate problems
- As a result, one characteristic separating truly skilled programmer from novices.

Why the course is important

- Algorithms as a technology
- Suppose computers were infinitely fast and memory was free, is there any need to study algorithm?
- Need to find the easiest one to implement.
- May be very fast, but not infinitely.
- Memory may be very cheap, but not free.
- The resources (cpu time and memory) need to be used wisely, (efficient) algorithms will help.

- As the basis of computer science discipline.
- Core course in computer science/engineering department
- (Put emphasis on practical aspects of algorithms).

Insertion Sort Algorithm

Input: a sequence of n number a1, a2, …,an

Output: a permutation (reordering) a1', a2', …,an'

such that a1' a2' … an '.

- Idea: (similar to sort a hand of playing cards)
- Every time, take one card, and insert the card to correct position in already sorted cards.

Insertion Sort Algorithm (cont.)

INSERTION-SORT(A)

- forj = 2 to length[A]
- dokey A[j]
- //insert A[j] to sorted sequence A[1..j-1]
- i j-1
- whilei >0and A[i]>key
- do A[i+1] A[i] //move A[i] one position right
- i i-1
- A[i+1] key

Example of Insertion Sort

Correctness of Insertion Sort Algorithm

- Loop invariant
- At the start of each iteration of the for loop, the subarray A[1..j-1] contains original A[1..j-1] but in sorted order.

- Proof:
- Initialization : j=2, A[1..j-1]=A[1..1]=A[1], sorted.
- Maintenance: each iteration maintains loop invariant.
- Termination: j=n+1, so A[1..j-1]=A[1..n] in sorted order.

- Pseudocode conventions

Analyzing algorithms

- Time (not memory, bandwidth, hardware)
- Implementation model
- Random-Access-Model (RAM)
- One processor, sequential execution, no concurrency.
- Basic data types
- Basic operations (constant time per operation)

- (Parallel multi-processor access model: PRAM)

- Random-Access-Model (RAM)

Analysis of Insertion Sort

INSERTION-SORT(A) costtimes

- forj = 2 to length[A] c1n
- dokey A[j] c2n-1
- //insert A[j] to sorted sequence A[1..j-1] 0n-1
- i j-1 c4n-1
- whilei >0and A[i]>keyc5j=2n tj
- do A[i+1] A[i] c6j=2n(tj –1)
- i i-1 c7j=2n(tj –1)
- A[i+1] key c8n–1
(tj is the number of times the while loop test in line 5 is executed for that value of j)

The total time cost T(n) = sum of cost times in each line

=c1n + c2(n-1) + c4(n-1) + c5j=2n tj+ c6j=2n (tj-1)+ c7j=2n (tj-1)+ c8(n-1)

Analysis of Insertion Sort (cont.)

- Best case cost: already ordered numbers
- tj=1, and line 6 and 7 will be executed 0 times
- T(n) = c1n + c2(n-1) + c4(n-1) + c5(n-1) + c8(n-1)
=(c1 + c2 + c4 + c5 + c8)n – (c2 + c4 + c5 + c8) = cn + c‘

- Worst case cost: reverse ordered numbers
- tj=j,
- so j=2n tj = j=2n j =n(n+1)/2-1, and j=2n(tj –1) = j=2n(j–1) = n(n-1)/2, and
- T(n) = c1n + c2(n-1) + c4(n-1) + c5(n(n+1)/2 -1) + + c6(n(n-1)/2 -1) + c7(n(n-1)/2)+ c8(n-1) =((c5 + c6 + c7)/2)n2 +(c1 + c2 + c4 +c5/2-c6/2-c7/2+c8)n-(c2 + c4 + c5 + c8) =an2+bn+c

- Average case cost: random numbers
- in average, tj = j/2. T(n) will still be in the order of n2, same as worst case.

The Worst Case Running Time

- An upper bound for any input.
- For some algorithms, worst case occurs often.
- Average case is often roughly as bad as the worst case.

Order of growth

- Lower order item(s) are ignored, just keep the highest order item.
- The constant coefficient(s) are ignored.
- The rate of growth, or the order of growth, possesses the highest significance.
- Use (n2) to represent the worst case running time for insertion sort.
- (1), (lg n), (n),(n), (nlg n), (n2), (n3), (2n), (n!)

Design Approaches

- Recursive algorithm
- To solve a given problem, the algorithm recursively calls itself one or more time to deal with closely related subproblems.
- f(n) = 0 if n=1
1 if n=2

f(n-1) + f(n-2), if n>2

- Divide-and-conquer
- Dynamic programming
- Prune-and-search
- Greedy algorithms, Linear programming, Parallel algorithms, Approximate algorithms, etc.

Divide-and-conquer approach

- Divide the problem into a number of subproblems, with the same type as the original problem type.
- Conquer the subproblems by solving them recursively. If the size of a subproblem is small enough, just solve the subproblem directly.
- Combine the solutions to the subproblems into the solution for the original problem.

Merge Sort

- Divide: divide the n-element sequence into two subproblems of n/2 elements each.
- Conquer: sort the two subsequences recursively using merge sort. If the length of a sequence is 1, do nothing since it is already in order.
- Combine: merge the two sorted subsequences to produce the sorted answer.

Merge Sort –merge function

- Merge is the key operation in merge sort.
- Suppose the (sub)sequence(s) are stored in the array A. moreover, A[p..q] and A[q+1..r] are two sorted subsequences.
- MERGE(A,p,q,r) will merge the two subsequences into sorted sequence A[p..r]

MERGE(A,p,q,r)

- n1 q – p + 1
- n2 r – q
- //createtemporaryL[1..n1+1], R[1..n2+1]
- fori 1 ton1
- do L[i] A[p+i-1]
- forj 1 ton2
- do R[j] A[q+j]
- L[n1 +1]
- R[n2 +1]
- i 1
- j 1
- Fork ptor
- do if L[i] R[j]
- then A[k] L[i]
- i i+1
- else A[k] R[j]
- j j+1

Correctness of MERGE(A,p,q,r)

- Loop invariant
- At start of each iteration of for loop of line 12-17, subarray A[p..k-1] contains the k-p smallest elements of L[1..n1+1] and R[1..n2+1] in sorted order. L[i] and R[j] are smallest elements of their arrays that have not been copied back to A.

- Proof:
- Initialization: k-p=0, i=j=1, L[1] and R[1] are smallest.
- Maintenance: loop will maintain the invariant
- Termination: k=r+1, so A[p..k-1]= A[p..r] contains all and in sorted.

Analysis of MERGE(A,p,q,r)

- n1 q – p + 1 (1)
- n2 r – q (1)
- //createtemporaryL[1..n1+1], R[1..n2+1] (0)
- fori 1 ton1 (n1) (n1 +1 times)
- do L[i] A[p+i-1] (1)* (n1) (n1 times)
- forj 1 ton2 (n2) (n2 +1 times)
- do R[j] A[q+j] (1) * (n2) (n2 times)
- L[n1 +1] (1)
- R[n2 +1] (1)
- i 1 (1)
- j 1 (1)
- Fork ptor (n), suppose n = r-p+1
- do if L[i] R[j] (1)*(n)
- then A[k] L[i] (1)*O(n)
- i i+1 (1)*O(n)
- else A[k] R[j] (1)*O(n)
- j j+1 (1)*O(n)
Thus the total running time is (n1)+ (n2)+ (n)+7 (1) = (n).

MERGE-SORT(A,p,r)

- ifp < r
- thenq (p+r)/2
- MERGE-SORT(A,p,q)
- MERGE-SORT(A,q+1,r)
- MERGE(A,p,q,r)
Call to MERGE-SORT(A,1,n) (suppose n=length(A))

Analysis of Divide-and-Conquer

- Described by recursive equation
- Suppose T(n) is the running time on a problem of size n.
- T(n) = (1) if nc
aT(n/b)+D(n)+C(n) if n>c

Where a: number of subproblems

n/b: size of each subproblem

D(n): cost of divide operation

C(n): cost of combination operation

Analysis of MERGE-SORT

- Divide: D(n) = (1)
- Conquer: a=2,b=2, so 2T(n/2)
- Combine: C(n) = (n)
- T(n) = (1) if n=1
2T(n/2)+ (n) if n>1

- T(n) = c if n=1
2T(n/2)+ cn if n>1

Compute T(n) by Recursive Tree

- The recursive equation can be solved by recursive tree.
- T(n) = 2T(n/2)+ cn, (See its Recursive Tree).
- lg n+1 levels, cn at each level, thus
- Total cost for merge sort is
- T(n) =cnlg n +cn = (nlg n).

- In contrast, insertion sort is
- T(n) = (n2).

Recursion tree of T(n)=2T(n/2)+cn

Growth of functions

- Asymptotic efficiency:
- Input size is very large
- Ignore lower-order terms and constant coefficients
- Only the highest order item, (order of growth).

- Typical order of growth:
- (1), (lg n), (n),(n), (nlg n), (n2), (n3), (2n), (n!)

- Asymptotic notations: O, , , o, .

-notation

- For a given function g(n),
- (g(n))={f(n): there exist positive constants c1,c2,and n0 such that 0 c1g(n) f(n) c2g(n) for all n n0}
- Write f(n) (g(n)), or simply f(n) = (g(n)).

c2g(n)

f(n)

c1g(n)

n

n0

f(n) = (g(n))

Some (g(n)) functions

- Prove 1/2n2 – 3n = (n2).
- c1 n2 1/2n2 – 3n c2 n2
- c1 1/2 – 3/n c2
- c2 ½ for any n 1, and c1 1/14 for any n 7
- Thus, c1= 1/14, c2 = ½ , n0=7.

- Prove 6n3 (n2).
- In general, f(n)=an2+bn+c = (n2), where a,b,c are constants and a>0.
- p(n) = i=0daini, where ai are constants and ad>0.
- then p(n) = (nd).

- (n0) or (1), constant time complexity.

O-notation

- For a given function g(n),
- O(g(n))={f(n): there exist positive constants c and n0 such that 0 f(n) cg(n) for all n n0}
- Write f(n) O(g(n)), or simply f(n) = O(g(n)).

cg(n)

f(n)

n

n0

f(n) = O(g(n))

Relation between (g(n)) and O(g(n))

- f(n) = (g(n)) implies f(n) = O(g(n))
- -notation is a stronger notion than O-notation.
- (g(n)) O(g(n))
- an2+bn+c = (n2), also =O(n2)
- A little surprising: an+b = O(n2). Verify it!!!
- Even n =O(n2) !!!
- O-notation, represents asymptotic upper bound.
- -notation, represents asymptotic tight bound.
- Note for distinctions:
- When we say worst-case running time for insertion sort is O(n2), which applies to all inputs, even an already sorted input runs only in O(n) time because O(n) O(n2).
- However, worst-case running time (n2) for insertion sort does not apply to every input, since for an already sorted input, (n) (n2) .

-notation

- For a given function g(n),
- (g(n))={f(n): there exist positive constants c and n0 such that 0 cg(n) f(n) for all n n0}
- Write f(n) (g(n)), or simply f(n) = (g(n)).

f(n)

cg(n)

n

n0

f(n) = (g(n))

Relation among O, ,-notations

- Theorem 3.1 (page 46)
- For f(n) and g(n), f(n) = (g(n)) if and only if f(n) = O(g(n)) and f(n) = (g(n)).

- O-notation: asymptotic upper bound.
- -notation: asymptotic tight bound.
- -notation: asymptotic lower bound.

Notes on -notation

- generally used for best case running time.
- So correct for all inputs.
- For example, in insertion sort
- best case running time is (n),
- or correctly say: running time is (n).
- It is OK to say worst case running time is (n2)
- However, it is incorrect to say running time is (n2).

o-notation

- For a given function g(n),
- o(g(n))={f(n): for any positive constant c,there exist a positive n0 such that 0 f(n) cg(n) for all n n0}
- Write f(n) o(g(n)), or simply f(n) = o(g(n)).

2g(n)

g(n)

1/2g(n)

f(n)

n

n0

n0

n0

f(n) = o(g(n))

g(n)

n

Notes on o-notition- O-notation may or may not be asymptotically tight for upper bound.
- 2n2 = O(n2) is tight, but 2n = O(n2) is not tight.

- o-notition is used to denote an upper bound that is not asymptotically tight.
- 2n = o(n2), but 2n2o(n2).

- Difference: for some positive constant c in O-notation, but all positive constants c in o-notation.
- In o-notation, f(n) becomes insignificant relative to g(n) as n approaches infinitely: i.e.,
- lim = 0.

f(n)

-notation

- For a given function g(n),
- (g(n))={f(n): for any positive constant c, there exists a positive n0 such that 0 cg(n) f(n) for all n n0}
- Write f(n) (g(n)), or simply f(n) = (g(n)).

- -notation, similar to o-notation, denotes lower bound that is not asymptotically tight.
- n2/2 = (n), but n2/2 (n2)

- f(n) = (g(n)) if and only if g(n)=o(f(n)).
- lim =

f(n)

g(n)

n

Properties of asymptotic notations

- Transitivity: all five notations
- f(n)= (g(n)) and g(n)= (h(n)) f(n)= (h(n))

- Reflexivity: O, , notations.
- f(n)= (f(n))

- Symmetry: notation.
- f(n)= (g(n)) if and only if g(n)= (f(n))

- Transpose symmetry:
- f(n) = O(g(n)) if and only if g(n)= (f(n)).
- f(n) = o(g(n)) if and only if g(n)= (f(n)).

Asymptotic Efficiency of Recurrences

- Find the asymptotic bounds of recursive equations.
- Substitution method
- Recursive tree method
- Master method (master theorem)
- Provides bounds for: T(n) = aT(n/b)+f(n) where
- a 1 (the number of subproblems).
- b>1, (n/b is the size of each subproblem).
- f(n) is a given function.

- Provides bounds for: T(n) = aT(n/b)+f(n) where

Recurrences

- MERGE-SORT
- T(n) = (1) if n=1
2T(n/2)+ (n) if n>1

- Ignore certain details:
- T(n) = (1) if n=1
T(n/2)+ T(n/2)+ (n) if n>1

- Moreover, T(n) = 2T(n/2)+ (n)

- T(n) = (1) if n=1

- T(n) = (1) if n=1

The Substitution Method

- Two steps:
- Guess the form of the solution.
- By experience, and creativity.
- By Some heuristics.
- If a recurrence is similar to one you have seen before.
- T(n)=2T(n/2+17)+n, similar to T(n)=2T(n/2)+n, , guess O(nlg n).

- Prove loose upper and lower bounds on the recurrence and then reduce the range of uncertainty.
- For T(n)=2T(n/2)+n, prove lower bound T(n)= (n), and prove upper bound T(n)= O(n2), then guess the tight bound is T(n)=O(nlg n).

- If a recurrence is similar to one you have seen before.
- By recursion tree.

- Use mathematical induction to find the constants and show that the solution works.

- Guess the form of the solution.

Solve T(n)=2T(n/2)+n

- Guess the solution: T(n)=O(nlg n),
- i.e., T(n)cnlg n for some c.

- Prove the solution by induction:
- Suppose this bound holds for n/2, i.e.,
- T(n/2)cn/2lg (n/2).

- T(n) 2(cn/2lg (n/2))+n
- cnlg (n/2))+n
- = cnlg n - cnlg 2 +n
- = cnlg n - cn +n
- cnlg n (as long as c1)

- Suppose this bound holds for n/2, i.e.,

- Is the above proof complete? Why?

Boundary (base) Condition

- In fact, T(n) =1 if n=1, i.e., T(1)=1.
- However, cnlg n =c1lg 1 = 0, which is odd with T(1)=1.
- Take advantage of asymptotic notation: it is required T(n)cnlg n hold for n n0 where n0 is a constant of our choosing.
- Select n0 =2, thus, n=2 and n=3 as our induction bases. It turns out any c 2 suffices for base cases of n=2 and n=3 to hold.

Subtleties

- Guess is correct, but induction proof not work.
- Problem is that inductive assumption not strong enough.
- Solution: revise the guess by subtracting a lower-order term.
- Example: T(n)=T(n/2)+T(n/2)+1.
- Guess T(n)=O(n), i.e., T(n) cn for some c.
- However, T(n) c n/2+c n/2+1 =cn+1, which does not imply T(n) cn for any c.
- Attempting T(n)=O(n2) will work, but overkill.
- New guess T(n) cn – b will work as long as b 1.

Avoiding Pitfall

- It is easy to guess T(n)=O(n) (i.e., T(n) cn) for T(n)=2T(n/2)+n.
- And wrongly prove:
- T(n) 2(cn/2)+n
- cn+n
- =O(n). wrongly !!!!

- T(n) 2(cn/2)+n
- Problem is that it does not prove the exactform of T(n) cn.

Changing Variables

- Suppose T(n)=2T(n)+lg n.
- Rename m=lg n. so T(2m)=2T(2m/2)+m.
- Rename S(m)=T(2m), so S(m)=2S(m/2)+m.
- Which is similar to T(n)=2T(n/2)+n.
- Sothe solution is S(m)=O(m lg m).
- Changing back to T(n) from S(m), the solution is T(n) =T(2m)=S(m)=O(m lg m)=O(lg n lg lg n).

The Recursion-tree Method

- Idea:
- Each node represents the cost of a single subproblem.
- Sum up the costs with each level to get level cost.
- Sum up all the level costs to get total cost.

- Particularly suitable for divide-and-conquer recurrence.
- Best used to generate a good guess, tolerating “sloppiness”.
- If trying to compute cost as exact as possible, then used as direct proof.

cn2

c(n/4)2

c(n/4)2

c(n/4)2

T(n/16)

T(n/16)

T(n/16)

T(n/16)

T(n/16)

T(n/16)

T(n/16)

T(n/16)

T(n/16)

(c)

Recursion Tree for T(n)=3T(n/4)+(n2)T(n)

cn2

T(n/4)

T(n/4)

T(n/4)

(a)

(b)

cn2

cn2

(3/16)cn2

c(n/4)2

c(n/4)2

c(n/4)2

log 4n

(3/16)2cn2

c(n/16)2

c(n/16)2

c(n/16)2

c(n/16)2

c(n/16)2

c(n/16)2

c(n/16)2

c(n/16)2

c(n/16)2

(nlog 43)

T(1)

T(1)

T(1)

T(1)

T(1)

T(1)

nlog 43

Total O(n2)

(d)

Solution to T(n)=3T(n/4)+(n2)

- The height is log 4n,
- #leaf nodes = 3log 4n= nlog 43. Leaf node cost: T(1).
- Total cost T(n)=cn2+(3/16) cn2+(3/16)2cn2+
…+(3/16)log 4n-1cn2+ (nlog 43)

=(1+3/16+(3/16)2+…+(3/16)log 4n-1) cn2 + (nlog 43)

<(1+3/16+(3/16)2+…+(3/16)m+…) cn2 + (nlog 43)

=(1/(1-3/16)) cn2 + (nlog 43)

=16/13cn2 + (nlog 43)

=O(n2).

Prove the above Guess

- T(n)=3T(n/4)+(n2) =O(n2).
- Show T(n) dn2 for some d.
- T(n) 3(d (n/4)2) +cn2
3(d (n/4)2) +cn2

=3/16(dn2) +cn2

dn2, as long as d(16/13)c.

One more example

- T(n)=T(n/3)+ T(2n/3)+O(n).
- Construct its recursive tree (Figure 4.2, page 71).
- T(n)=O(cnlg3/2n) = O(nlg n).
- Prove T(n) dnlg n.

Recursion Tree of T(n)=T(n/3)+ T(2n/3)+O(n)

Master Method/Theorem

- Theorem 4.1 (page 73)
- for T(n) = aT(n/b)+f(n), n/b may ben/born/b.
- where a 1, b>1, f(n) be a function.
- If f(n)=O(nlogba-) for some >0, then T(n)= (nlogba).
- If f(n)= (nlogba), then T(n)= (nlogba lg n).
- If f(n)=(nlogba+) for some >0, and if af(n/b) cf(n) for some c<1 and all sufficiently large n, then T(n)= (f(n)).

Implications of Master Theorem

- Comparison between f(n) and nlogba (<,=,>)
- Must be asymptotically smaller (or larger) by a polynomial, i.e., n for some >0.
- In case 3, the “regularity” must be satisfied, i.e., af(n/b) cf(n) for some c<1 .
- There are gaps
- between 1 and 2: f(n) is smaller than nlogba, but not polynomially smaller.
- between 2 and 3: f(n) is larger than nlogba, but not polynomially larger.
- in case 3, if the “regularity” fails to hold.

Application of Master Theorem

- T(n) = 9T(n/3)+n;
- a=9,b=3, f(n) =n
- nlogba = nlog39 = (n2)
- f(n)=O(nlog39-) for =1
- By case 1, T(n) = (n2).

- T(n) = T(2n/3)+1
- a=1,b=3/2, f(n) =1
- nlogba = nlog3/21 = (n0) = (1)
- By case 2, T(n)= (lg n).

Application of Master Theorem

- T(n) = 3T(n/4)+nlg n;
- a=3,b=4, f(n) =nlg n
- nlogba = nlog43 = (n0.793)
- f(n)= (nlog43+) for 0.2
- Moreover, for large n, the “regularity” holds for c=3/4.
- af(n/b) =3(n/4)lg(n/4) (3/4)nlg n =cf(n)

- By case 3, T(n) = (f(n))= (nlg n).

Exception to Master Theorem

- T(n) = 2T(n/2)+nlg n;
- a=2,b=2, f(n) =nlg n
- nlogba = nlog22 = (n)
- f(n) is asymptotically larger than nlogba , but not polynomially larger because
- f(n)/nlogba = lg n, which is asymptotically less than n for any >0.
- Therefore,this is a gap between 2 and 3.

Where Are the Gaps

f(n), case 3, at least polynomially larger

n

Gap between case 3 and 2

c1

f(n), case 2: within constant distances

nlogba

c2

n

Gap between case 1 and 2

f(n), case 1, at least polynomially smaller

Note: 1. for case 3, the regularity also must hold.

2. if f(n) is lg n smaller, then fall in gap in 1 and 2

3. if f(n) is lg n larger, then fall in gap in 3 and 2

4. if f(n)=(nlogbalgkn), then T(n)=(nlogbalgk+1n). (as exercise)

logbn-1

j=0

Proof of Master Theorem- The proof for the exact powers, n=bk for k1.
- Lemma 4.2
- for T(n) = (1) if n=1
- aT(n/b)+f(n) if n=bk for k1
- where a 1, b>1, f(n) be a nonnegative function,
- Then
- T(n) = (nlogba)+ ajf(n/bj)

- Proof:
- By iterating the recurrence
- By recursion tree (See figure 4.3)

Recursion tree for T(n)=aT(n/b)+f(n)

logbn-1

j=0

Proof of Master Theorem (cont.)- Lemma 4.3:
- Let a 1, b>1, f(n) be a nonnegative function defined on exact power of b, then
- g(n)= ajf(n/bj) can be bounded for exact power of b as:
- If f(n)=O(nlogba-) for some >0, then g(n)= O(nlogba).
- If f(n)= (nlogba), then g(n)= (nlogba lg n).
- If af(n/b) cf(n) for some c<1 and all sufficiently large n b, then g(n)= (f(n)).

logbn-1

logbn-1

logbn-1

logbn-1

logbn-1

j=0

j=0

j=0

j=0

j=0

Proof of Lemma 4.3- For case 1: f(n)=O(nlogba-) implies f(n/bj)=O((n /bj)logba-), so
- g(n)=ajf(n/bj) =O(aj(n /bj)logba- )
- = O(nlogba- aj/(blogba-)j ) = O(nlogba- aj/(aj(b-)j))
- = O(nlogba- (b)j ) = O(nlogba- (((b )logbn-1)/(b-1) )
- = O(nlogba- (((blogbn)-1)/(b-1)))=O(nlogba n-(n-1)/(b-1))
- = O(nlogba )

logbn-1

logbn-1

logbn-1

logbn-1

j=0

j=0

j=0

j=0

Proof of Lemma 4.3(cont.)- For case 2: f(n)= (nlogba) implies f(n/bj)= ((n /bj)logba), so
- g(n)=ajf(n/bj) = (aj(n /bj)logba)
- = (nlogbaaj/(blogba)j ) = (nlogba1)
- = (nlogba logbn) = (nlogbalg n)

logbn-1

logbn-1

j=0

j=0

Proof of Lemma 4.3(cont.)- For case 3:
- Since g(n) contains f(n), g(n) = (f(n))
- Since af(n/b) cf(n), ajf(n/bj) cjf(n) , why???
- g(n)= ajf(n/bj) cjf(n) f(n)cj
- =f(n)(1/(1-c)) =O(f(n))
- Thus, g(n)=(f(n))

j=0

Proof of Master Theorem (cont.)

- Lemma 4.4:
- for T(n) = (1) if n=1
- aT(n/b)+f(n) if n=bk for k1
- where a 1, b>1, f(n) be a nonnegative function,
- If f(n)=O(nlogba-) for some >0, then T(n)= (nlogba).
- If f(n)= (nlogba), then T(n)= (nlogba lg n).
- If f(n)=(nlogba+) for some >0, and if af(n/b) cf(n) for some c<1 and all sufficiently large n, then T(n)= (f(n)).

Proof of Lemma 4.4 (cont.)

- Combine Lemma 4.2 and 4.3,
- For case 1:
- T(n)= (nlogba)+O(nlogba)=(nlogba).

- For case 2:
- T(n)= (nlogba)+(nlogba lg n)=(nlogba lg n).

- For case 3:
- T(n)= (nlogba)+(f(n))=(f(n)) because f(n)= (nlogba+).

- For case 1:

Floors and Ceilings

- T(n)=aT(n/b)+f(n) and T(n)=aT(n/b)+f(n)
- Want to prove both equal T(n)=aT(n/b)+f(n)
- Two results:
- Master theorem applied to all integers n.
- Floors and ceilings does not change the result.

- Since n/bn/b, and n/b n/b, upper bound for floors and lower bound for ceiling is held.
- So prove upper bound for ceilings (similar for lower bound for floors).

Upper bound of proof for T(n)=aT(n/b)+f(n)

- consider sequence n, n/b, n/b/b, n/b/b/b, …
- Let us define nj as follows:
- nj = n if j=0
- = nj-1/bif j>0
- The sequence will be n0, n1, …, nlogbn
- Draw recursion tree:

Recursion tree of T(n)=aT(n/b)+f(n)

logbn-1

j=0

The proof of upper bound for ceiling- T(n) = (nlogba)+ ajf(nj)
- Thus similar to Lemma 4.3 and 4.4, the upper bound is proven.

Summary

- Analysis of running time and space
- Worst case running time.
- Average case (expected) running time.
- Amortized analysis and amortized cost.

- Best cast running time.

- Loop invariant and its proof.
- Asymptotic notations, definitions and relations.
- Recurrences and their bounds
- Substitution
- Recursion tree
- Master theorem.
- Proof of subtleties and recurrence Master theorem does not apply to.

Download Presentation

Connecting to Server..