1 / 37

380 likes | 615 Views

Fundamentals of Algorithm Analysis. Algorithm : Design & Analysis [Tutorial - 1]. Qian Zhuzhong （钱柱中）. Research: Distributed Computing Pervasive Computing Service Oriented Computing Office: 503A, MMW Email: qianzhuzhong@dislab.nju.edu.cn. In Previous Classes….

Download Presentation
## Fundamentals of Algorithm Analysis

**An Image/Link below is provided (as is) to download presentation**
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.
Content is provided to you AS IS for your information and personal use only.
Download presentation by click this link.
While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

**Fundamentals of Algorithm Analysis**Algorithm : Design & Analysis [Tutorial - 1]**Qian Zhuzhong （钱柱中）**• Research: Distributed Computing • Pervasive Computing • Service Oriented Computing • Office: 503A, MMW • Email: qianzhuzhong@dislab.nju.edu.cn**In Previous Classes…**Introduction to Algorithm Analysis Asymptotic Behavior of Functions Recursion and Master Theorem Sorting**In Tutorial One**About the Tutorial Algorithm Analysis Revisiting Asymptotic Behavior Revisiting Recursion**About the Tutorial**• The course • Algorithm design and analysis • Coverage • The tutorial: Reemphasize important issues by • Further explanation • Typical examples • Interaction • …**Algorithm Analysis**Solve problems Find efficient solutions Naïve solution Better solutions optimal solution … efficiency * http://cs.nju.edu.cn/yuhuang/huangyufiles/alg/computational_thinking.pdf • Before learning algorithm analysis • Learn to solve problems by“computational thinking”* • Data structures • After leaning algorithm analysis • How efficient is my first solution? • How to improve? • Better solutions • The optimal solution**Asymptotic Behavior**• Discussion of asymptotic notations • Some properties • An example: Maximum Subsequence Sum • Improvement of Algorithm • Comparison of Asymptotic Behavior**The definition of O, and **• O • Giving g:N→R+, then Ο(g) is the set of f:N→R+, such that for some cR+ and some n0N, f(n)cg(n) for all nn0. • A function fΟ(g) if limn→[f(n)/g(n)]=c< • • Giving g:N→R+, then (g) is the set of f:N→R+, such that for some cR+ and some n0N, f(n)cg(n) for all nn0. • A function f(g) if limn→[f(n)/g(n)]=c>0 • • Giving g:N→R+, then (g) = Ο(g) (g) • A function f(g) if limn→[f(n)/g(n)]=c, 0<c<**“little Oh” and ω**• o • Giving g:N→R+, then o(g) is the set of f:N→R+, such that for anycR+ and some n0N, f(n)cg(n) for all nn0. • A function fo(g) if limn→[f(n)/g(n)]=c=0 • ω • Giving g:N→R+, then ω(g) is the set of f:N→R+, such that for any cR+ and some n0N, f(n)cg(n) for all nn0. • A function fω(g) if limn→[f(n)/g(n)]=c=**Analogy**fΟ(g) ≈ a ≤ b f(g) ≈ a ≥ b f(g) ≈ a = b fo(g) ≈ a < b fω(g) ≈ a > b**Properties of O(o), (ω) and **• Transitive property(O,, ,ω, o): • If fO(g) and gO(h), then fO(h), … • Reflexive property: • f(n)(f(n)), f(n)O(f(n)), f(n)(f(n)) • Symmetric properties • f(g) if and only if g(f) • fO(g) if and only if g(f) • fo(g) if and only if gω(f) • Order of sum function • O(f+g)=O(max(f, g))**Maximum Subsequence Sum**A brute-force algorithm: MaxSum = 0; for (i = 0; i < N; i++) for (j = i; j < N; j++) { ThisSum = 0; for (k = i; k <= j; k++) ThisSum += A[k]; if (ThisSum > MaxSum) MaxSum = ThisSum; } return MaxSum; the sequence j=0 j=1 j=2 j=n-1 i=0 i=1 k i=2 …… in O(n3) i=n-1 • The problem: Given a sequence S of integer, find the largest sum of a consecutive subsequence of S. (0, if all negative items) • An example: -2, 11, -4, 13, -5, -2; the result 20: (11, -4, 13)**Decreasing the number of loops**An improved algorithm MaxSum = 0; for (i = 0; i < N; i++) { ThisSum = 0; for (j = i; j < N; j++) { ThisSum += A[j]; if (ThisSum > MaxSum) MaxSum = ThisSum; } } return MaxSum; the sequence i=0 i=1 j i=2 in O(n2) i=n-1**Part 1**Part 1 Part 2 Part 2 Power of Divide-and-Conquer the sub with largest sum may be in: recursion or: The largest is the result Part 1 Part 2 in O(nlogn)**Divide-and-Conquer: the Procedure**Center = (Left + Right) / 2; MaxLeftSum = MaxSubSum(A, Left, Center); MaxRightSum = MaxSubSum(A, Center + 1, Right); MaxLeftBorderSum = 0; LeftBorderSum = 0; for (i = Center; i >= Left; i--) { LeftBorderSum += A[i]; if (LeftBorderSum > MaxLeftBorderSum) MaxLeftBorderSum = LeftBorderSum; } MaxRightBorderSum = 0; RightBorderSum = 0; for (i = Center + 1; i <= Right; i++) { RightBorderSum += A[i]; if (RightBorderSum > MaxRightBorderSum) MaxRightBorderSum = RightBorderSum; } return Max3(MaxLeftSum, MaxRightSum, MaxLeftBorderSum + MaxRightBorderSum); Note: this is the core part of the procedure, with base case and wrap omitted.**A Linear Algorithm**ThisSum 0 0 -2 -1 0 4 10 2 -3 0 2 5 4 2 11 MaxSum 0 0 0 0 4 4 10 10 10 10 10 10 10 10 11 -2 -1 4 6 -8 -5 2 2 3 3 -1 -1 -2 -2 9 9 the sequence ThisSum = MaxSum = 0; for (j = 0; j < N; j++) { ThisSum += A[j]; if (ThisSum > MaxSum) MaxSum = ThisSum; else if (ThisSum < 0) ThisSum = 0; } return MaxSum; j This is an example of “online algorithm” in O(n) Negative item or subsequence cannot be a prefix of the subsequence we want.**Recursion**• Problem solving • Divide and conquer • Recurrence equation • Solve the recurrence • Characteristic Equation • Master Theorem • How do we obtain the results? • Rationale behind the detailed mathematical proof**External Path Length**• The external path length of a 2-treet is defined as follows: • The external path length of a leaf, which is a 2-tree consisting of a single external node, is 0 • If t is a nonleaf 2-tree, with left subtree L and right subtree R, then the external path length of t is the sum of: • the external path length of L; • the number of external node of L; • the external path length of R; • the number of external node of R; • In fact, the external path length of t is the sum of the lengths of all the paths from the root of t to any external node in t.**2-Tree**Common Binary Tree 2-Tree internal nodes Both left and right children of these nodes are empty tree external nodes no child any type**Calculating the External Path Length**TwoTree is an ADT defined for 2-tree EplReturn is a organizer class with two field epl and extNum EplReturn calcEpl(TwoTree t) EplReturn ansL, ansR; EplReturn ans=new EplReturn(); 1. if (t is a leaf) 2. ans.epl=0; ans.extNum=1; 3. else 4. ansL=calcEpl(leftSubtree(t)); 5. ansR=calcEpl(rightSubtree(t)); 6. ans.epl=ansL.epl+ansR.epl+ansL.extNum +ansR.extNum; 7. ans.extNum=ansL.extNum+ansR.extNum 8. Return ans;**Correctness of Procedure calcEpl**• Let t be any 2-tree. Let epl and m be the values of the fields epl and extNum, respectively, as returned by calcEpl(t). Then: • 1. epl is the external path length of t. • 2. m is the number of external nodes in t. • 3. eplmlg(m) (note: for 2-tree with internal n nodes, m=n+1)**Proof on Procedure calcEpl**• Induction on t, with the “subtree” partial order: • Base case: t is a leaf. (line 2) • Inductive hypothesis: the 3 statements hold for any proper subtree of t, say s. • Inductive case: by ind. hyp., eplL, eplR, mL, mR,are expected results for L and R(both are proper subtrees of t), so: • Statement 1 is guranteed by line 6 • Statement 2 is guranteed by line 7 (any external node is in either L or R) • Statement 3: by ind.hyp. epl=eplL+eplR+mmLlg(mL)+mRlg(mR)+m, note f(x)+f(y)2f((x+y)/2) if f is convex, and xlgxis convex for x>0, so, epl 2((mL+mR)/2)lg((mL+mR)/2)+m = m(lg(m)-1)+m =mlgm.**Characteristic Equation**If the characteristic equation of the recurrence relation has two distinct roots s1 and s2, then where u and v depend on the initial conditions, is the explicit formula for the sequence.**Number of Valid Strings**• String to be transmitted on the channel • Length n • Consisting of symbols ‘a’, ‘b’, ‘c’ • If “aa” exists, cannot be transmitted • E.g. strings of length 2: ‘ab’, ‘ac’, ‘ba’, ‘bb’, ‘bc’, ‘ca’, ‘cc’, ‘cb’ • Number of valid strings?**Divide and conquer**b c n-1 n-1 a b a c n-2 n-2 • f(n)=2f(n-1)+2f(n-2), n>2 • f(1)=3, f(2)=8**Characteristic equation**Solution Analysis of the D&C solution**Recursion Tree for**T(n)=bT(n/c)+f(n) f(n) f(n) b f(n/c) f(n/c) f(n/c) b logcn f(n/c2) f(n/c2) f(n/c2) f(n/c2) f(n/c2) f(n/c2) f(n/c2) f(n/c2) f(n/c2) …… …… … … T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) Note: Total ?**Divide-and-Conquer: the Solution**• The recursion tree has depth D=lg(n)/ lg(c), so there are about that many row-sums. • The solution of divide-and-conquer equation is the nonrecursive costs of all nodes in the tree, which is the sum of the row-sums. • The 0th row-sum is f(n), the nonrecursive cost of the root. • The Dth row-sum is nE, assuming base cases cost 1, or (nE) in any event.**Solution by Row-sums**This can be generalized to get a result not using explicitly row-sums. • [Little Master Theorem] Row-sums decide the solution of the equation for divide-and-conquer: • Increasing geometric series: T(n)(nE) • Constant: T(n)(f(n) log n) • Decreasing geometric series: T(n)(f(n))**The positive is critical, resulting gaps between cases**as well Master Theorem • Loosening the restrictions on f(n) • Case 1: f(n)O(nE-), (>0), then: T(n)(nE) • Case 2: f(n)(nE), as all node depth contribute about equally: T(n)(f(n)log(n)) • case 3: f(n)(nE+), (>0), and f(n)O(nE+), (), then: T(n)(f(n))**Looking at the Gap**• T(n)=2T(n/2)+nlgn • a=2, b=2, E=1, f(n)=nlgn • We have f(n)=(nE), but no >0 satisfies f(n)=(nE+), since lgn grows slower that n for any small positive . • So, case 3 doesn’t apply. • However, neither case 2 applies. • Why is important?**Standard Algorithm – by definition**Run time = Θ (n3)**Divide-and-conquer Algorithm**Idea: n*n matrix = 2*2 of (n/2) * (n/2) sub-matrices:**Analysis of the D&C Algorithm**# sub-matrices Adding sub-matrices Sub-matrix size

More Related