Algorithms

1. Algorithms Joe Meehean

2. Data Structures vs. Algorithms • Data structures are great at… • storing items • providing a simple access interface • Algorithms … • operate over data structures • instructions to do more complicated work • may or may not want embedded in data structure

3. Algorithms in STL • Found in <algorithm> header • From simple • for_each • find • swap • To the complex • sort • set_intersection • random_shuffle • next_permutation

4. Brute Force • straight forward approach • usually based directly on • problem statement • definitions of concepts • e.g., an • an = a * a * …. * a (n times)

5. Brute Force • Advantages • applicable to a wide variety of problems • no limitation on problem size for some important problems • simple to design • designing better algo not always worth it • if problem small or algo will run infrequently • provides a comparison for more complex algos

6. Brute Force • Disadvantages • slow • may be so slow as to make impossible to complete in human lifetime

7. Sorting • Problem • arrange comparable items in list into sorted order • Most sorting algorithms involve comparing item values • We assume items define • < operator • > operator • == operator

8. Selection Sort Idea Find the smallest value in vector A and put in it in A[0] Find 2nd smallest value and put it in A[1] Etc…

9. Selection Sort Approach • Use a nested loop • Outer loop index k • indicates position to fill • Inner loop index j • from k+1 to A.length – 1 • indicates value to compare to min next • Swap A[k] with A[min] • A[min] is min value in range k+1 to A.length–1

10. Selection Sort Approach min 8 2 9 0 5 6 3 4 1 7 k

11. Selection Sort Approach min 8 2 9 0 5 6 3 4 1 7 k j

12. Selection Sort Approach min 8 2 9 0 5 6 3 4 1 7 k j

13. Selection Sort Approach min 8 2 9 0 5 6 3 4 1 7 k j

14. Selection Sort Approach min 8 2 9 0 5 6 3 4 1 7 k j

15. Selection Sort Approach min 8 2 9 0 5 6 3 4 1 7 k j

16. Selection Sort Approach min 8 2 9 0 5 6 3 4 1 7 k j

17. Selection Sort Approach min 8 2 9 0 5 6 3 4 1 7 k j

18. Selection Sort Approach min 8 2 9 0 5 6 3 4 1 7 k j

19. Selection Sort Approach min 8 2 9 0 5 6 3 4 1 7 k j

20. Selection Sort Approach swap min 8 2 9 0 5 6 3 4 1 7 k j

21. Selection Sort Approach swap min 0 2 9 8 5 6 3 4 1 7 k j

22. Selection Sort Approach min 0 2 9 8 5 6 3 4 1 7 k j

23. Selection Sort Approach

24. Selection Sort Item value Position in the Array

25. Section Sort Discussion • After i outer loop iterations • A[0] through A[i-1] contain their final values

26. Time for Selection Sort • Outer loop executes N times • Inner loop executes a different number of times depending on outer loop • 1st outer = N – 1 inner • 2nd outer = N – 2 inner • … • Nth outer = 0 inner • (N-1) + (N-2) +…+ 2 + 1 + 0 = O(N2) • Always O(N2)

27. Exhaustive Search • Combinatorial problems • problems where the answer is an ordered combination of items from a set • Exhaustive search • brute force approach to combinatorial problems • generate all possible combinations • check each combination to see if it is a possible solution • may need to find the best solution • e.g., try all possible combinations to a safe

28. 0-1 Knapsack Problem • Have a set of items • each has a weight: wi • each has a monetary value: vi • Have knapsack • can only hold a total weight of W • Fill the knapsack to maximize its value

29. 0-1 Knapsack Problem • Exhaustive search • try every combination of items • throw out combinations that are too heavy • keep the combination with the largest value • Complexity • dominated by generating all combinations • each item may be in the knapsack or not • for N items: 2N possible combinations • O(2N): very, very bad

30. Questions?

31. Decrease & Conquer Exploit relationship between solution to a problem and solution to a smaller instance Reduce problem to a smaller problem Solve smaller problem Use smaller problem solution to solve original problem

32. Decrease & Conquer • Top-down • reduce larger problem into successively smaller problems • recursive approach • Bottom-up • solve smallest version of problem • use to solve next larger problem • incremental approach

33. Decrease & Conquer • 3 variations • Decrease-by-constant • compute an for positive integer n • an = an-1 * a • f(n) = an • f(n) = f(n-1) * a, if n >0 • f(n) = 1, if n == 0 • recursive definition

34. Decrease & Conquer • 3 variations • Decrease-by-constant-factor • an = (an/2)2 • n/2 is not an integer if n is odd, so… • an = (an/2)2, if n is even • an = (a(n-1)/2)2 * a, if n is odd • an= 1, if n = 0 • O(logN) number of multiplications

35. Decrease & Conquer • 3 variations • Variable-size-decrease • lookup in BST • BST is unbalanced • at each node going left or right removes a variable number of nodes

36. Insertion Sort Idea Incremental approach Reduce list size to trivial, sort and solve Then increase list size Put 1st 2 items in correct order Insert the 3rd item in the correct place relative to the first 2 Insert the 4th item in the correct place relative to the first 3 etc…

37. Insertion Sort Approach • Nested loop • Outer loop • index k from 1 to A.length – 1 • Item to put into correct place • Inner loop • index j from k – 1 to 0 • Items to compare to A[k] to find its correct place

38. Insertion Sort 8 2 9 5 6 3 4 1 0 7 j k

39. Insertion Sort temp 5 8 2 9 6 3 4 1 0 7 j k

40. Insertion Sort temp 5 2 9 6 3 4 8 1 0 7 j k

41. Insertion Sort 2 9 6 3 4 5 8 1 0 7 j k

42. Insertion Sort 5 2 9 8 6 3 4 1 0 7 j k

43. Insertion Sort temp 2 5 9 8 6 3 4 1 0 7 j k

44. Insertion Sort temp 2 5 9 6 3 4 1 0 8 7 j k

45. Insertion Sort temp 2 5 9 6 3 4 1 0 8 7 j k

46. Insertion Sort 5 9 6 3 4 1 0 8 2 7 j k

47. Insertion Sort 5 9 6 3 4 1 0 8 2 7 j k

48. Insertion Sort temp 6 5 9 3 4 1 0 8 2 7 j k

49. Insertion Sort temp 6 5 9 3 4 1 0 8 2 7 j k

50. Insertion Sort 5 9 3 4 1 0 6 8 2 7 j k