Chapter 5: Decrease and Conquer

1. The Design and Analysis of Algorithms Chapter 5:Decrease and Conquer

2. Chapter 5. Decrease and Conquer Algorithms • Basic Idea • Decrease by a Constant (usually one) • Insertion sort, graph search • Permutations, subsets • Decrease by a Constant Factor • Fake-coin problem, Multiplication a la Russe • Variable Size Decrease • Conclusion

3. Basic Idea • Reduce problem instance to smaller instance of the same problem and extend solution • Solve smaller instance • Extend solution of smaller instance to obtain solution to original problem

4. Basic Idea

5. Examples of Decrease-and-Conquer Algorithms • Decrease by one: • Insertion sort • Graph search algorithms: • DFS • BFS • Topological sorting • Algorithms for generating permutations, subsets

6. Examples of Decrease-and-Conquer Algorithms • Decrease by one: • Insertion sort • Graph search algorithms: • DFS • BFS • Topological sorting • Algorithms for generating permutations, subsets

7. Examples of Decrease-and-Conquer Algorithms • Decrease by a constant factor • Binary search • Fake-coin problems • Multiplication à la russe • Josephus problem • Variable-size decrease • Euclid’s algorithm • Selection by partition

8. Decrease by One: Insertion Sort Insertion Sort Algorithm to sort n elements: • Sort n-1 elements of the array • Insert the n-th element Complexity: (n2) in the worst and the average case, and (n) on almost sorted arrays.

9. Decrease by One: Graph Search • Depth-first search algorithm: • dfs(v) • process(v) • mark v as visited • for all vertices i adjacent to v not visited • dfs(i) Complexity:Θ(V2) if represented by an adjacency table, Θ(|V| + |E|) if represented by adjacency lists

10. Graph Search: DFS

11. Graph Search: BFS Breadth-first search algorithm: procedure bfs(v) q := make_queue() enqueue(q,v) mark v as visited while q is not empty v = dequeue(q) process v for all unvisited vertices v' adjacent to v mark v' as visited enqueue(q,v')

12. Graph Search: BFS

13. Graph Search • Complexity of DFS and BFS:Θ(V2) if represented by an adjacency table, Θ(|V| + |E|) if represented by adjacency lists. • Various applications in AI problems and graph problems (e.g. finding cycles, articulation points)

14. Permutations of Size n Generating permutations of size n • find all permutations of size n-1 of elements a1, a2, .., a n-1 • construct permutations of n elements as: • append an to each permutation of size n-1 • for each permutation of size n-1 for k from 1 to n-1 insert an in front of ak

15. Johnson-Trotter Algorithm • The straight-forward implementation is very inefficient since it obtains all lower-level permutations (permutations of size less than n). • Trotter (1962) and Johnson (1963): an algorithm to obtain all permutations of size n without going through shorter permutations. • Directed integers. • Mobile integer: greater than the integer it points to

16. Johnson-Trotter Algorithm Initialize the first permutation with <1 <2 ... <n while the last permutation has a mobile integer do find the largest mobile integer k swap k and the adjacent integer it is looking at reverse the direction of all integers larger than k

17. Johnson-Trotter Algorithm Example: <1 <2 <3 <4 largest mobile element is 4 <1 <2 <4 <3 swap 3 and 4; largest mobile element is 4 <1 <4 <2 <3 swap 2 and 4; largest mobile element is 4 <4 <1 <2 <3 swap 1 and 4; largest mobile element is 3 4> <1 <3 <2 swap 2 and 3; change direction of all greater than 3; largest mobile element is 4 <1 4> <3 <2 swap 1 and 4; largest mobile element is 4 <1 <3 4> <2 swap 3 and 4; largest mobile element is 4 <1 <3 <2 4> swap 2 and 4; largest mobile element is 3 . . . . . . . . <2 <1 3> 4>. “minimal-change” algorithm; runs in (N!) time

18. Subsets and Gray Codes Let Sn-1 be the set of all subsets of n-1 elements, Sn-1 = {A1, A2, … Am}, m = 2n-1 Sn = {A1, A2, … Am, A1  an , A2  an , … Am  an } No need to generate all power sets of smaller sets. Frank Gray (1953): a minimal-change algorithm for generating all binary sequences of length n - “binary reflected Gray code”.

19. Subsets and Gray Codes base reflect prepend reflect prepend 0 0 00 00 000 1 101 01 001 111 11 011 010 10010 10110 11111 01101 00100 The complexity is (2n)

20. Decrease by a Constant Factor • The Fake-coin problem • Multiplication a la Russe Usually logarithmic in complexity

21. The Fake Coin Problem You have 27 coins among which one is fake and it is lighter than the others. You have also a balance scale and you can compare the weight of any two sets of coins. How can you find the fake coin with three measurements only? How many measurements if the number of coins is N?

22. Multiplications a la Russe We can multiply two positive integers using only addition and division by 2. The algorithm is based on the observation that N*M = (N/2) * (M * 2) if N is even, and N*M = ((N-1)/2) * (M*2) + M if N is odd The base case is N = 1: 1*M = M

23. Multiplications a la Russe

24. Variable Size Decrease • The reduction pattern varies from one iteration of the algorithm to another. • Examples are • Euclid’s algorithm for computing the greatest common divisor , • Search and Insertion in binary search trees, and others.

25. Conclusion • Very efficient solutions. • The algorithms exploit a relation between the instance of the problem and a smaller instance of the same problem • Recursive in nature • Implementation: recursion or iteration