Chapter 1: Introduction to Algorithm Design

1. Chapter 1: Introduction to Algorithm Design Dr. Paige Meeker CSC 336

2. What is an Algorithm? • In order to know how to create an algorithm and to understand the importance of algorithms, we should know what they are. So, what are they?

3. An Algorithm is… • The idea behind a computer program. • Stays the same, no matter what language is used to create the code. • Should solve a general, specified problem • So, what’s a problem?

4. A Problem is… • Specified by describing the set of instances on which an algorithm must work and what properties are desired of the output. • NOTE: A problem and an instance of a problem are two very different things!

5. Example Problem: Sorting • The algorithmic problem known as sorting is defined as follows:

6. Example Problem: Sorting • An instance of the sorting problem might be one of the following: • An array of names • A list of numbers • A group of records

7. First Steps! • In class, you will be given problems to solve. Discovering the general problem behind the instance given to you will lead you quite a way down the path toward solving the problem.

8. Correct and Efficient • Solutions must be correct. For any algorithm, we must prove it always produces the desired output for all legal instances of the problem • Example: Sorting – must provide sorted output even if • The input is already sorted • It contains repeated elements • You have an empty list • Etc…

9. Correct and Efficient • Efficient we’ll talk about more later – let’s just say that no one likes to wait too long to get results for now… • We also don’t want to strain the resources of our machines too badly – no one likes the dreaded blue screen of death.

10. A Sorting Algorithm (there are many!) • Insertion Sort – anyone know how it works?

11. Insertion Sort • Works the way people sort a handful of playing cards. Take an empty left hand and the cards face down on the table. • Pick up one card at a time and place them into their appropriate position in your hand, comparing each new card with the ones in your hand from left to right until the right spot is found. • At all times, the cards in the left hand are sorted

12. What else makes a good algorithm design? • It should be easy to implement. (Didn’t think I’d say that, did you?) • You may not be able to accomplish the three goals of design (correct, efficient, easy to implement) every time; however, they are what we’re shooting for.

13. Next problem: Robot Tour Optimization

15. Possible Solutions? • Nearest Neighbor • Start at some point and then walk to the nearest neighbor point. Repeat until done.

16. Nearest Neighbor

17. Advantages to Nearest Neighbor… • Simple to understand and implement • Makes sense • Works on previous example • Any problems?

18. Nearest Neighbor Won’t Work! • Doesn’t work in all instances of the problem. • While it always finds a tour, doesn’t always find the shortest one.

19. What about always starting at the leftmost point? • Which is the leftmost point now? • THUS: You are working to solve the GENERAL problem, not a particular INSTANCE of the problem.

20. So, now what? • Ok – so we tried one thing and it didn’t work. Let’s say that always choosing the closest point is too restrictive – • Try to repeatedly connect the closest pair of points whose connection will not create a problem (cause a cycle or a three-way branch) until all points are in one tour.

21. Closest Pair Tour • Works GREAT for the last example… any problems?

22. Closest Pair Tour • Consider a set of two rows of equally spaced points, with the rows slightly closer together than the neighboring points are spaced in each row.

23. How about just finding all paths and picking the best one? (aka: Exhaustive Search) • Guaranteed to end up with the shortest tour. • Correct – we pick the best of all possibilities • REALLY SLOW! 20 points would produce 20! (2,432,902,008,176,640,000) orderings • Not even realistic, as most circuit boards have about 1,000 points to deal with. The end of time may come before this algorithm solves the problem! (and then, why would we need a solution? • This is also called the Traveling Salesman Problem – heard of it now?

24. Take Home Lesson • There is a fundamental difference between algorithms which always produce a correct result and heuristics, which may usually do a good job but without providing any guarantee.

25. Why not just use a better computer? • A faster algorithm running on a slower computer will always win for sufficiently large instances of a problem. • Usually, problems don’t have to get that large before the faster algorithm wins.

26. Expressing Algorithms • We need to be able to express the sequence of steps used in an algorithm in order to pass that information along to others. • In order of increasing precision, we have English, Pseudocode, and Real Programming Languages (like Java or C++) • Unfortunately, ease of expression moves in the reverse order!

27. Expressing Algorithms • English is the most natural but least precise way to express a description of an algorithm. If you use English descriptions, you must be very careful to describe fully each step to be performed.

28. Expressing Algorithms • Programming languages like Java, C, or C++ are precise ways of expressing algorithms, but difficult to write and understand

29. Expressing Algorithms • Pseudocode is best defined as a programming language that never complains about syntax errors.

30. Expressing Algorithms • What am I looking for? • Describe the ideas of the algorithm in English • Use Pseudocode or a programming language to clarify, in a formal way, the tricky details you are concerned you may have missed in the English description.

31. Take Home Lesson • The heart of any algorithm is an idea. If your idea is not clearly revealed when you express an algorithm, they you are using too low-level a notation to describe it.

32. Problem Specifications • You can expect problems to be specified in two parts: • The set of allowable input instances • The required properties of the algorithms output.

33. Problem Specifications • Common Traps in Problem Specifications • Sometimes, problem specifications allow too broad a class of input instances – it is your job to state any assumptions you make about the input and narrow it down to something that can be reasonably solved. • Asking ill-defined questions for the output (find the “best” path – best should be defined – shortest, fastest, minimizing the number of turns, etc.) • Don’t create compound goals for the algorithm – there should be a single criteria.

34. Take Home Lesson • An important and honorable technique in algorithm design is to narrow the set of allowable instances until there is a correct and efficient algorithm.

35. Next Problem…

36. Stated more formally: Input: A set I of n intervals on the line Output: What is the largest subset of mutually non-overlapping intervals which can be selected from I? Your mission? Give an algorithm to solve the problem! Ideas???

37. Ideas? • Should work whenever work is available – earliest job first. • Take the shortest job available first, so that I’m available more often. • Try every possible permutation and see what works best! • First job to terminate.

38. Earliest job first • Start working as soon as there is work available. More formally: EarliestJobFirst(I) Accept the earliest starting job j from I which does not overlap any previously accepted job, and repeat until no such jobs remain.

39. Earliest Job first won’t work! • The first job might be so long (War and Peace) that it prevents us from taking any other job.

40. Shortest Job First • Always take the shortest possible job, so you spend the least time working (and thus the least time unavailable) SortestJobFirst(I) While (I != 0) do Accept the shortest possible job j from I Delete j, and intervals which intersect j from I

41. Shortest Job First Won’t Work! • Taking the shortest job can prevent us from taking two longer jobs which barely overlap it.

42. Exhaustive Scheduling • REALLY SLOW! – can be done, but takes 2n where n is the number of jobs • Better than robot optimization (n! ) as there are only about a million subsets to deal with; still...

43. Does a solution exist? • YES!! Think about the first job to terminate – other jobs may have started before that job, but all of these must partially overlap each other and we can select at most one from this group. Any overlappers could potentially block new jobs, so we can’t loose by choose this one!

44. First Job to Complete • Take the job with the earliest completion date OptimalScheduling(I) While (I != 0) do Accept job j with the earliest completion date Delete j and whatever intersects j from I

45. Take Home Lesson • Reasonable Looking algorithms can easily be incorrect. Algorithm correctness is a property that must be carefully demonstrated.

46. Demonstrating Incorrectness • Searching for counterexamples is the best way to disprove the correctness of a heuristic • Think about all small examples • Think about examples with ties on your design criteria (pick the nearest point) • Think about examples with extremes of big and small

47. Hunting for Counterexamples • Think Small – several small examples are easier to understand and verify than one large ugly one. When algorithms fail, there is usually a very simple example which causes it to happen.

48. Hunting for Counterexamples • Look for weaknesses – if the algorithms says “always take the biggest” think about ways that wouldn’t work – what if the input was all the same size? Suddenly, the algorithm has no way to choose a “good” starting point and may choose a less than optimal one.

49. Hunting for Counterexamples • Look for extremes – Many counterexamples are mixtures of huge and tiny, few and many, near and far. It is usually easier to verify or reason about extremes.

50. Take Home Lesson • Searching for counterexamples is the best way to disprove the correctness of a heuristic.