算法设计与分析 Algorithm design and Analysis

1. 精研课程 算法设计与分析 Algorithm design and Analysis 叶 德 仕 计算机学院 yedeshi@zju.edu.cn

2. 课程基本信息 • 课程编号： 21190120 • 上课时间、地点：2013年 夏学期 • 周二(1~2)曹光彪西101 、周四（1~2）曹光彪西101 • 考试时间：7月3日 闭卷

3. Office & Homepage • Office：工商楼 215 • My Homepage：http://www.cs.zju.edu.cn/people/yedeshi/

4. Examination • Grading Polices： • Homework: 10% • Quiz: 15% (May 21) • Presentation: 15% • Programming Projects: 20% (2 programms) 5) Final Exam: 40%

5. TA information • 史如意: roles • Guide to group the students • Collect the homework, program, and project • Review of homework, programs • Email: shiruyi3@gmail.com

6. Algorithms Programming

7. The core of TCS • Algorithms and complexity of computation • Computational limits of proof methods • Logic and program verification • The power of randomization • Cryptography • Quantum computation • Distributed computation and communication • Computational learning theory

8. Important themes in TCS • Efficiency common measures: computation time, memory, parallelism, randomness,.. • Impossibility resultsintellectual ancestors: impossibility of perpetual motion, impossibility of trisecting an angle, incompleteness theorem, undecidability, etc. • Approximationapproximately optimal answers, algorithms that work “most of the time”,mathematical characterizations that are approximate (e.g., approximatemax-flow min-cut theorem) • Central role of randomnessrandomized algorithms and protocols, probabilistic encryption,random graph models, probabilistic models of the WWW, etc. • ReductionsNP-completeness and other intractability results (including complexity-based cryptography)

9. What is the position of algorithms in CS • 1. Linguists: what shall we talk to the machines? • 2. Algorithms: what is a good method for solving a problem fast on my computer • 3. Architects: Can I build a better computer? • 4. Sculptors of Machine Intelligence: Can I write a computer program that can find its own solution.

10. Algorithms in Computer Science Hardware Algorithms Compilers, Programming languages Networking, Distributed systems, Fault tolerance, Security Machine learning, Statistics, Information retrieval, AI Bioinformatics .......

11. MIT Undergraduate Programs

12. What is algorithm？ • （Oxford Dict.）Algorithm: • A set of rules that must be followed when solving a particular problem. • From Math world • A specific set of instructions for carrying out a procedure or solving a problem, usually with the requirement that the procedure terminate at some point. • An algorithm is any well-defined computational procedure that takes some value, or set of values, as input and produces some value, or set of values, as output.

13. Algorithm • Problem definition 问题 • Objective 目标 (very important) • Evaluation 算法评价 • Methods 方法

14. Algorithm evaluation • Quality: • how far away from the optimal solution ? • Cost: • Running time • Space needed • Our goal is to design algorithm with high quality, but in low cost

15. Reasonable times • Poly(|I|), Time polynomial in |I|, where |I| is the size of the problem instance • Input size: size(x) of an instance x with rational data is the total number of bits needed for the binary prepresentation.

16. Time complexity • logarithmic time if T(n) = O(log n). • sub-linear time if T(n) = o(n) • linear time, or O(n) time • linearithmic function: T(n) = O(n log n), quasilinear time if T(n) = O(n logkn) • polynomial time: T(n) = O(nk) for some constant k • strongly polynomial time: • the number of operations in the arithmetic model of computation is bounded by a polynomial in the number of integers in the input instance; and • the space used by the algorithm is bounded by a polynomial in the size of the input. • weakly polynomial time: P but not strongly P

17. Time complexity • Quasi-polynomial time: for some fixed c. • Sub-exponential time if T(n) = 2o(n) • Exponential time, if T(n) is upper bounded by 2poly(n)

18. Hardness of problems Easy • Polynomial (e.g. n2, n log n, n3, n1000). • Quasi-polynomial(e.g.:n log n, n log2n,c log7n). • Sub-exponential (e.g.: 2√n, 5(n0.98)). • Exponential (e.g.: 2n, 8n, n!, nn). Hard

19. Running time • Computer A is 100 times faster than computer B • Sort n numbers • Computer A requires instructions • Computer B requires 50nlgninstructions • n = 1,000, 000 • Computer A: 2(10^6)^2/10^9 = 2000 seconds • Computer B: 50*10^6 lg 10^6/10^7 ~ 100 seconds

20. 2 3 6 2 5 l ! n 1 2 1 0 0 n n n n n o g n Running time

21. Sorting • 输入：A sequence of n number • 输出：排列（permutation ） < > … a a a 2 1 n , , , 0 0 0 < > … a a a 1 2 , , , n 使得： 0 0 0 <= <= <= ... a a a 1 2 n Example: Input:8 2 4 9 3 6 Output:2 3 4 6 8 9

22. EX. of insertion sort 8 2 4 9 3 6

23. 8 2 4 9 3 6 EX. of insertion sort 2 8 4 9 3 6

24. 8 2 4 9 3 6 2 8 4 9 3 6 EX. of insertion sort

25. 8 2 4 9 3 6 2 8 4 9 3 6 EX. of insertion sort 2 4 8 9 3 6

26. 8 2 4 9 3 6 2 8 4 9 3 6 2 4 8 9 3 6 EX. of insertion sort

27. 8 2 4 9 3 6 2 8 4 9 3 6 2 4 8 9 3 6 EX. of insertion sort 2 4 8 9 3 6

28. 8 2 4 9 3 6 2 8 4 9 3 6 2 4 8 9 3 6 EX. of insertion sort 2 4 8 9 3 6

29. 8 2 4 9 3 6 2 8 4 9 3 6 2 4 8 9 3 6 EX. of insertion sort 2 4 8 9 3 6 2 3 4 8 9 6

30. 8 2 4 9 3 6 2 8 4 9 3 6 2 4 8 9 3 6 EX. of insertion sort 2 4 8 9 3 6 2 3 4 8 9 6

31. 8 2 4 9 3 6 2 8 4 9 3 6 2 4 8 9 3 6 EX. of insertion sort 2 4 8 9 3 6 2 3 4 8 9 6 2 3 4 6 8 9 done

32. INSERTION-SORT (A, n) ⊳ A[1 . . n] forj ← 2ton dokey ← A[ j] i ← j – 1 while i > 0 and A[i] > key doA[i+1] ← A[i] i ← i – 1 A[i+1] = key “pseudocode” Insertion sort 1 i j n A: key sorted

33. Analyzing algorithms • Need a computational model • Random-access machine (RAM) model • Instructions are executed one after another. No concurrent operations. • Arithmetic: add, subtract, multiply, divide, remainder, floor, ceiling • Data movement: load, store, copy • Control: conditional/unconditional branch, subroutine call and return. • Each of these instructions takes a constant amount of time.

34. Running time • Running time: • The running time of an algorithm on a particular input is the number of primitive operations or “steps” executed. • line consists only of primitive operations and takes constant time • Input size: • number of items • the total number of bits. • more than one number: Graph • the number of vertices and the • number of edges

35. Example: • The input size of sorting problem is n.Worst-case running time of Insert sort is O(n2).

36. Running time • The running time depends on the input: an already sorted sequence is easier to sort. • Parameterize the running time by the size of the input, since short sequences are easier to sort than long ones. • Generally, we seek upper bounds on the running time, because everybody likes a guarantee.

37. Map of Algorithm Design New problem Off-line problem On-line problem Polynomial Polynomial NP-C problem Quality Appro. ratio Exact Algorithm Approximate Algorithm Heuristic Improve cost running time Improve cost running time Quality Appro. ratio

38. 课程内容 • 1.数学基础 • 1.1 算法基础 • 1.2 和 (SUMS) 集合运算 (Sets) • 1.3 特殊数 （Stirling numbers, Harmonic numbers, Eulerian numbers et al.） • 2.基本算法 • 2.1分治（Divide-and-Conquer）* • 2.1.1 Mergesort * • 2.1.2 自然数相乘（Multiplication）* • 2.1.3 矩阵相乘（Matrix multiplication） • 2.1.4 Discrete Fourier transform and Fast Fourier transform

39. 课程内容 • 2.2 动态规划 (Dynamic Programming) • 2.2.1 背包问题（Knapsack problem） • 2.2.2 最长递增子序列（Longest increasing subsequence） • 2.2.3 Sequence alignment • 2.2.4 最长相同子序列（Longest common subsequence） • 2.3.5 Matrix-chain multiplication • 2.3.6 树上的独立集 (Max Independent set in tree)

40. 课程内容 • 2.3 贪婪算法 （Greedy） • 2.3.1 区间规划（Interval scheduling） • 2.3.2 集合覆盖（Set cover） • 2.3.3 拟阵（Matroids） • 2.4 NP 问题 （NP-completeness） • 2.4.1 The classes P and NP • 2.4.2 NP-completeness and reducibility • 2.4.3 NP-complete problems *

41. 课程内容 • 2.5 近似算法 (Approximate Algorithm) • 2.5.1 顶点覆盖问题 （Vertex cover） • 2.5.2 负载平衡问题 (Load balancing) • 2.5.3 旅行商问题 (Traveling salesman problem) • 2.5.4 子集和问题 (Subset sum problem)

42. 课程内容 • 3. 算法的应用 • 3.1 局部搜索 (Local Search) • 3.1.1 The Metropolis Algorithm and Simulated Annealing • 3.1.2 Local Search to Hopfield Neural Networks（Nash Equilibria） • 3.1.3 Maximum Cut Approximation via Local Search

43. 课程内容 • 3.2 图论 (Graph Theorem) * • 3.2.1 图论的基本知识 （Fundamental） • 3.2.2 线性规划 (Linear Programming) • 网络流（Network Flow），二分图，完全图的匹配 • 3.3计算几何学 (Computational Geometry)* • 3.3.1 基本概念与折线段的性质 (Line-segment ) • 3.3.2 线段的一些性质 (Segments intersects ) • 3.3.3 凸包问题 (Convex Hull ) • 3.3.4 最近点对问题 (The closet pair of points) • 3.3.5 多边形三角剖分 (Polygon Triangulation)

44. 课程内容 • 3.4 随机算法 (Randomized Algorithm) • 3.4.1 随机变量与期望 • 3.4.2 A Randomized MAX-3-SAT • 3.4.3 Randomized Divide-and-Conquer • 3.5 在线算法（Online Algorithm） • 3.5.1 Online Skying • 3.5.2 Online Hiring *：备选内容

45. 课程内容

46. 教材 • Textbook: Introduction to algorithms, Second Edition. Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest and Clifford Stein. The MIT Press, 2001. ISBN: 0262032937. • Recommended: Algorithm Design.Jon Kleinberg, Éva Tardos. Addison Wesley, 2005. ISBN: 0-321-29535-8. Rolf Nevanlinna Prize, 06

48. 参考教材 • Algorithms.S. Dasgupta, C.H. Papadimitriou, and U. V. Vazirani.May 2006. • Combinatorial Algorithms. Jeff Erickson. University of Illinois, Urbana-Champaign. Lecture Notes. Fall 2002. • Concrete Mathematics. Ronald L. Graham, Donald E. Knuth, Oren Patashnik. Addison-Wesley Publishing Company, 2005. ISBN: o-201-14236-8.

49. Algorithms in Computer Science • P = NP? • Can we solve a problem efficiently? • Tradeoff between quality of solution and the running time • Solve a problem with optimal solution, but it might cost long time • Solve a problem approximately in short time

50. $1,000,000 problem • P = NP? http://www.claymath.org/millennium/ • Solved???!!!!