176 Views

Download Presentation
##### Fundamentals of Algorithm Analysis

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Fundamentals of Algorithm Analysis**Algorithm : Design & Analysis [Tutorial - 1]**Qian Zhuzhong （钱柱中）**• Research: Distributed Computing • Pervasive Computing • Service Oriented Computing • Office: 503A, MMW • Email: qianzhuzhong@dislab.nju.edu.cn**In Previous Classes…**Introduction to Algorithm Analysis Asymptotic Behavior of Functions Recursion and Master Theorem Sorting**In Tutorial One**About the Tutorial Algorithm Analysis Revisiting Asymptotic Behavior Revisiting Recursion**About the Tutorial**• The course • Algorithm design and analysis • Coverage • The tutorial: Reemphasize important issues by • Further explanation • Typical examples • Interaction • …**Algorithm Analysis**Solve problems Find efficient solutions Naïve solution Better solutions optimal solution … efficiency * http://cs.nju.edu.cn/yuhuang/huangyufiles/alg/computational_thinking.pdf • Before learning algorithm analysis • Learn to solve problems by“computational thinking”* • Data structures • After leaning algorithm analysis • How efficient is my first solution? • How to improve? • Better solutions • The optimal solution**Asymptotic Behavior**• Discussion of asymptotic notations • Some properties • An example: Maximum Subsequence Sum • Improvement of Algorithm • Comparison of Asymptotic Behavior**The definition of O, and **• O • Giving g:N→R+, then Ο(g) is the set of f:N→R+, such that for some cR+ and some n0N, f(n)cg(n) for all nn0. • A function fΟ(g) if limn→[f(n)/g(n)]=c< • • Giving g:N→R+, then (g) is the set of f:N→R+, such that for some cR+ and some n0N, f(n)cg(n) for all nn0. • A function f(g) if limn→[f(n)/g(n)]=c>0 • • Giving g:N→R+, then (g) = Ο(g) (g) • A function f(g) if limn→[f(n)/g(n)]=c, 0<c<**“little Oh” and ω**• o • Giving g:N→R+, then o(g) is the set of f:N→R+, such that for anycR+ and some n0N, f(n)cg(n) for all nn0. • A function fo(g) if limn→[f(n)/g(n)]=c=0 • ω • Giving g:N→R+, then ω(g) is the set of f:N→R+, such that for any cR+ and some n0N, f(n)cg(n) for all nn0. • A function fω(g) if limn→[f(n)/g(n)]=c=**Analogy**fΟ(g) ≈ a ≤ b f(g) ≈ a ≥ b f(g) ≈ a = b fo(g) ≈ a < b fω(g) ≈ a > b**Properties of O(o), (ω) and **• Transitive property(O,, ,ω, o): • If fO(g) and gO(h), then fO(h), … • Reflexive property: • f(n)(f(n)), f(n)O(f(n)), f(n)(f(n)) • Symmetric properties • f(g) if and only if g(f) • fO(g) if and only if g(f) • fo(g) if and only if gω(f) • Order of sum function • O(f+g)=O(max(f, g))**Maximum Subsequence Sum**A brute-force algorithm: MaxSum = 0; for (i = 0; i < N; i++) for (j = i; j < N; j++) { ThisSum = 0; for (k = i; k <= j; k++) ThisSum += A[k]; if (ThisSum > MaxSum) MaxSum = ThisSum; } return MaxSum; the sequence j=0 j=1 j=2 j=n-1 i=0 i=1 k i=2 …… in O(n3) i=n-1 • The problem: Given a sequence S of integer, find the largest sum of a consecutive subsequence of S. (0, if all negative items) • An example: -2, 11, -4, 13, -5, -2; the result 20: (11, -4, 13)**Decreasing the number of loops**An improved algorithm MaxSum = 0; for (i = 0; i < N; i++) { ThisSum = 0; for (j = i; j < N; j++) { ThisSum += A[j]; if (ThisSum > MaxSum) MaxSum = ThisSum; } } return MaxSum; the sequence i=0 i=1 j i=2 in O(n2) i=n-1**Part 1**Part 1 Part 2 Part 2 Power of Divide-and-Conquer the sub with largest sum may be in: recursion or: The largest is the result Part 1 Part 2 in O(nlogn)**Divide-and-Conquer: the Procedure**Center = (Left + Right) / 2; MaxLeftSum = MaxSubSum(A, Left, Center); MaxRightSum = MaxSubSum(A, Center + 1, Right); MaxLeftBorderSum = 0; LeftBorderSum = 0; for (i = Center; i >= Left; i--) { LeftBorderSum += A[i]; if (LeftBorderSum > MaxLeftBorderSum) MaxLeftBorderSum = LeftBorderSum; } MaxRightBorderSum = 0; RightBorderSum = 0; for (i = Center + 1; i <= Right; i++) { RightBorderSum += A[i]; if (RightBorderSum > MaxRightBorderSum) MaxRightBorderSum = RightBorderSum; } return Max3(MaxLeftSum, MaxRightSum, MaxLeftBorderSum + MaxRightBorderSum); Note: this is the core part of the procedure, with base case and wrap omitted.**A Linear Algorithm**ThisSum 0 0 -2 -1 0 4 10 2 -3 0 2 5 4 2 11 MaxSum 0 0 0 0 4 4 10 10 10 10 10 10 10 10 11 -2 -1 4 6 -8 -5 2 2 3 3 -1 -1 -2 -2 9 9 the sequence ThisSum = MaxSum = 0; for (j = 0; j < N; j++) { ThisSum += A[j]; if (ThisSum > MaxSum) MaxSum = ThisSum; else if (ThisSum < 0) ThisSum = 0; } return MaxSum; j This is an example of “online algorithm” in O(n) Negative item or subsequence cannot be a prefix of the subsequence we want.**Recursion**• Problem solving • Divide and conquer • Recurrence equation • Solve the recurrence • Characteristic Equation • Master Theorem • How do we obtain the results? • Rationale behind the detailed mathematical proof**External Path Length**• The external path length of a 2-treet is defined as follows: • The external path length of a leaf, which is a 2-tree consisting of a single external node, is 0 • If t is a nonleaf 2-tree, with left subtree L and right subtree R, then the external path length of t is the sum of: • the external path length of L; • the number of external node of L; • the external path length of R; • the number of external node of R; • In fact, the external path length of t is the sum of the lengths of all the paths from the root of t to any external node in t.**2-Tree**Common Binary Tree 2-Tree internal nodes Both left and right children of these nodes are empty tree external nodes no child any type**Calculating the External Path Length**TwoTree is an ADT defined for 2-tree EplReturn is a organizer class with two field epl and extNum EplReturn calcEpl(TwoTree t) EplReturn ansL, ansR; EplReturn ans=new EplReturn(); 1. if (t is a leaf) 2. ans.epl=0; ans.extNum=1; 3. else 4. ansL=calcEpl(leftSubtree(t)); 5. ansR=calcEpl(rightSubtree(t)); 6. ans.epl=ansL.epl+ansR.epl+ansL.extNum +ansR.extNum; 7. ans.extNum=ansL.extNum+ansR.extNum 8. Return ans;**Correctness of Procedure calcEpl**• Let t be any 2-tree. Let epl and m be the values of the fields epl and extNum, respectively, as returned by calcEpl(t). Then: • 1. epl is the external path length of t. • 2. m is the number of external nodes in t. • 3. eplmlg(m) (note: for 2-tree with internal n nodes, m=n+1)**Proof on Procedure calcEpl**• Induction on t, with the “subtree” partial order: • Base case: t is a leaf. (line 2) • Inductive hypothesis: the 3 statements hold for any proper subtree of t, say s. • Inductive case: by ind. hyp., eplL, eplR, mL, mR,are expected results for L and R(both are proper subtrees of t), so: • Statement 1 is guranteed by line 6 • Statement 2 is guranteed by line 7 (any external node is in either L or R) • Statement 3: by ind.hyp. epl=eplL+eplR+mmLlg(mL)+mRlg(mR)+m, note f(x)+f(y)2f((x+y)/2) if f is convex, and xlgxis convex for x>0, so, epl 2((mL+mR)/2)lg((mL+mR)/2)+m = m(lg(m)-1)+m =mlgm.**Characteristic Equation**If the characteristic equation of the recurrence relation has two distinct roots s1 and s2, then where u and v depend on the initial conditions, is the explicit formula for the sequence.**Number of Valid Strings**• String to be transmitted on the channel • Length n • Consisting of symbols ‘a’, ‘b’, ‘c’ • If “aa” exists, cannot be transmitted • E.g. strings of length 2: ‘ab’, ‘ac’, ‘ba’, ‘bb’, ‘bc’, ‘ca’, ‘cc’, ‘cb’ • Number of valid strings?**Divide and conquer**b c n-1 n-1 a b a c n-2 n-2 • f(n)=2f(n-1)+2f(n-2), n>2 • f(1)=3, f(2)=8**Characteristic equation**Solution Analysis of the D&C solution**Recursion Tree for**T(n)=bT(n/c)+f(n) f(n) f(n) b f(n/c) f(n/c) f(n/c) b logcn f(n/c2) f(n/c2) f(n/c2) f(n/c2) f(n/c2) f(n/c2) f(n/c2) f(n/c2) f(n/c2) …… …… … … T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) Note: Total ?**Divide-and-Conquer: the Solution**• The recursion tree has depth D=lg(n)/ lg(c), so there are about that many row-sums. • The solution of divide-and-conquer equation is the nonrecursive costs of all nodes in the tree, which is the sum of the row-sums. • The 0th row-sum is f(n), the nonrecursive cost of the root. • The Dth row-sum is nE, assuming base cases cost 1, or (nE) in any event.**Solution by Row-sums**This can be generalized to get a result not using explicitly row-sums. • [Little Master Theorem] Row-sums decide the solution of the equation for divide-and-conquer: • Increasing geometric series: T(n)(nE) • Constant: T(n)(f(n) log n) • Decreasing geometric series: T(n)(f(n))**The positive is critical, resulting gaps between cases**as well Master Theorem • Loosening the restrictions on f(n) • Case 1: f(n)O(nE-), (>0), then: T(n)(nE) • Case 2: f(n)(nE), as all node depth contribute about equally: T(n)(f(n)log(n)) • case 3: f(n)(nE+), (>0), and f(n)O(nE+), (), then: T(n)(f(n))**Looking at the Gap**• T(n)=2T(n/2)+nlgn • a=2, b=2, E=1, f(n)=nlgn • We have f(n)=(nE), but no >0 satisfies f(n)=(nE+), since lgn grows slower that n for any small positive . • So, case 3 doesn’t apply. • However, neither case 2 applies. • Why is important?**Standard Algorithm – by definition**Run time = Θ (n3)**Divide-and-conquer Algorithm**Idea: n*n matrix = 2*2 of (n/2) * (n/2) sub-matrices:**Analysis of the D&C Algorithm**# sub-matrices Adding sub-matrices Sub-matrix size