UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Fall, 2008

1. UMass Lowell Computer Science 91.503Analysis of AlgorithmsProf. Karen DanielsFall, 2008 Lecture 2 Tuesday, 9/16/08 Design Patterns for Optimization Problems Greedy Algorithms

2. Algorithmic Paradigm Context Solve subproblem(s), then make choice Make choice, then solve subproblem(s) Subproblem solution order

3. Greedy Algorithms

4. What is a Greedy Algorithm? • Solves an optimization problem • Optimal Substructure: • optimal solution contains in it optimal solutions to subproblems • Greedy Strategy: • At each decision point, do what looks best “locally” • Choice does not depend on evaluating potential future choices or presolving overlapping subproblems • Top-down algorithmic structure • With each step, reduce problem to a smaller problem • Greedy Choice Property: • “locally best” = globally best

5. Greedy Strategy Approach • Determine the optimal substructure of the problem. • Develop a recursive solution. • Prove that, at any stage of the recursion, one of the optimal choices is the greedy choice. • Show that all but one of the subproblems caused by making the greedy choice are empty. • Develop a recursive greedy algorithm. • Convert it to an iterative algorithm. source: 91.503 textbook Cormen, et al.

6. Examples • Activity Selection • Minimum Spanning Tree • Dijkstra Shortest Path • Huffman Codes • Fractional Knapsack

7. Activity Selection

8. Activity Selection Optimization Problem • Problem Instance: • Set S = {1,2,...,n} of n activities • Each activity i has: • start time: si • finish time: fi • Activities i, j are compatible iff non-overlapping: • Objective: • select a maximum-sized set of mutually compatible activities source: 91.404 textbook Cormen, et al.

9. 1 2 4 5 9 3 12 16 6 10 14 15 11 7 13 8 1 2 3 4 5 6 7 8 Activity Selection Activity Time Duration Activity Number

10. Algorithmic Progression • “Brute-Force” • (board work) • Dynamic Programming #1 • Exponential number of subproblems • (board work) • Dynamic Programming #2 • Quadratic number of subproblems • Greedy Algorithm

11. Activity Selection Solution to Sij including ak produces 2 subproblems: 1) Sik (start after ai finishes; finish before ak starts) 2) Skj (start after ak finishes; finish before aj starts) c[i,j]=size of maximum-size subset of mutually compatible activities in Sij. source: 91.404 textbook Cormen, et al.

12. Recursive Activity Selection High-level call: RECURSIVE-ACTIVITY-SELECTOR(s,f,0,n) Returns an optimal solution for Errors from earlier printing are corrected in red. source: web site accompanying 91.503 textbook Cormen, et al.

13. source: web site accompanying 91.503 textbook Cormen, et al.

14. Running time? Greedy Algorithm source: 91.503 textbook Cormen, et al. • Algorithm: • S’ = presort activities in Sby nondecreasing finish time • and renumber • GREEDY-ACTIVITY-SELECTOR(S’) • n length[S’] • A {1} • j 1 • for i 2 to n • do if • then • ji • return A

15. Streamlined Greedy Strategy Approach • View optimization problem as one in which making choice leaves one subproblem to solve. • Prove there always exists an optimal solution that makes the greedy choice. • Show that greedy choice + optimal solution to subproblem optimal solution to problem. Greedy Choice Property: “locally best” = globally best source: 91.503 textbook Cormen, et al.

16. Minimum Spanning Tree

17. Invariant: Minimum weight spanning forest Becomes single tree at end Invariant: Minimum weight tree 2 A B 4 3 G 6 Spans all vertices at end 5 1 1 E 7 6 F 8 4 D C 2 Minimum Spanning Tree Time: O(|E|lg|E|) given fast FIND-SET, UNION • Produces minimum weight tree of edges that includes every vertex. Time: O(|E|lg|V|) = O(|E|lg|E|) slightly faster with fast priority queue • forUndirected, Connected, Weighted Graph • G=(V,E) source: 91.503 textbook Cormen et al.

18. Dijkstra Shortest Path

19. 2 A 4 3 6 5 1 1 7 B 6 F 8 G 4 C 2 E D Single Source Shortest Paths: Dijkstra’s Algorithm for (nonnegative)weighted,directed graph G=(V,E) source: 91.503 textbook Cormen et al.

20. Huffman Codes

21. Huffman Codes source: 91.503 textbook Cormen, et al.

22. source: web site accompanying 91.503 textbook Cormen, et al.

23. source: web site accompanying 91.503 textbook Cormen, et al.

24. source: web site accompanying 91.503 textbook Cormen, et al.

25. source: web site accompanying 91.503 textbook Cormen, et al.

26. source: web site accompanying 91.503 textbook Cormen, et al.

27. Fractional Knapsack

28. Knapsack Each item has value and weight. Goal: maximize total value of items chosen, subject to weight limit. 50 fractional: can take part of an item 0-1: take all or none of an item 30 20 10 item1 item2 item3 “knapsack” Value: $60 $100 $120

29. source: web site accompanying 91.503 textbook Cormen, et al.

30. Additional Examples On course web site under Miscellaneous Docs • Patriotic Tree • 404 review handout • Tree Vertex Cover • 91.503 midterm

31. Greedy Heuristic • If optimization problem does not have “greedy choice property”, greedy approach may still be useful as a heuristic in bounding the optimal solution • Example: minimization problem Upper Bound (heuristic) Solution Values Optimal (unknown value) Lower Bound

32. Homework HW# Assigned DueContent 1 T 9/9 T 9/16 91.404 review & Chapter 15 2 T 9/16 T 9/23 Chapter 16