1 / 24

Theory of Algorithms: Introduction

Theory of Algorithms: Introduction. James Gain and Edwin Blake {jgain | edwin} @cs.uct.ac.za Department of Computer Science University of Cape Town August - October 2004. Objectives. To define an algorithm To introduce: Problem types The Process of Algorithm Design Solution strategies

kwilfong
Download Presentation

Theory of Algorithms: Introduction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Theory of Algorithms:Introduction James Gain and Edwin Blake {jgain | edwin} @cs.uct.ac.za Department of Computer Science University of Cape Town August - October 2004

  2. Objectives • To define an algorithm • To introduce: • Problem types • The Process of Algorithm Design • Solution strategies • Ways of Analysing Algorithms • To cover the structure of the course, including practicals

  3. Definitions • An algorithm is a sequence of unambiguous instructions for solving a problem • For obtaining a required output for any legitimate input in a finite amount of time • Does not require implementation in software • Not an answer but a method for deriving an answer • Historical Perspective: • Named after Muhammad ibn Musa al-Khwarizmi – 9th century mathematician • www.lib.virginia.edu/science/parshall/khwariz.html

  4. Notion of algorithm problem algorithm “computer” input output • Each step of the algorithm must be unambiguous • The range of inputs must be specified carefully • The same algorithm can be represented in different ways • Several algorithms for solving the same problem may exist - with different properties

  5. What is an algorithm? • Recipe, process, method, technique, procedure, routine,… with following requirements: • Finiteness • Terminates after a finite number of steps • Definiteness • Rigorously and unambiguously specified • Input • Valid inputs are clearly specified • Output • Can be proved to produce the correct output given a valid input • Effectiveness • Steps are sufficiently simple and basic

  6. Example: Sorting • Statement of problem: • Input: A sequence of n numbers <a1, a2, …, an> • Output: A reordering of the input sequence <a´1, a´2, …, a´n> so that a´i≤ a´j whenever i < j • Instance: The sequence <5, 3, 2, 8, 3> • Algorithms: • Selection sort • Insertion sort • Merge sort • (many others)

  7. for i=1 to n swap a[i] with smallest of a[i],…,a[n] Selection Sort • Input: array a[1],..,a[n] • Output: array a sorted in non-decreasing order • Algorithm: • for i1 to ndo • min  i • for j  i+1 to n do • if a[j] < a[min] min  j • swap a[i] and a[min]

  8. Exercise: Bridge Puzzle • Problem: • 4 People want to cross a bridge. You have 17 minutes to get them across • Constraints: • It is night and you have 1 flashlight. Max of 2 on the bridge at one time. All start on the same side • Those crossing must have the flashlight with them. The flashlight must be walked back and forth (no throwing) • People walk at different speeds: person A = 1 minute to cross, person B = 2 minutes, person C = 5 minutes, person D = 10 minutes • A pair walks at the speed of the slower person’s pace • Rumour: this problem is given to Microsoft interviewees

  9. Solution: Bridge Puzzle • Start (0 min): A B C D • AB Across (2 min): A B C D • A Back (1 min): B A C D • CD Across (10 min): B C D A • B Back (2 min): C D A B • AB Across (2 min): A B C D • Total Time = 17 minutes

  10. Extension Exercise • This is an instance of a problem. How would you generalise it? • Can you derive an algorithm to solve this generalised problem? • Must show the sequence of moves • Must output the minimum time required for crossing • Are there any special cases to watch out for? • Are there any constraints on the input?

  11. Extension Solution • Input: a list a of crossing times for n people, numbered 1, …, n • Output: total time to cross • Strategy: use 1 & 2 as shuttles and send the others across in pairs • for i 2 to n/2do • t  a[2] // 1 & 2 across • t  t + a[1] // 1 back • t  t + a[i*2] // i*2 & (i*2)-1 across • t  t + a[2] // 2 back • t  a[2] // 1 & 2 across • return t

  12. Extension Problems • This is an inadequate solution • It falsely assumes certain inputs • List may not be sorted in ascending order • Sort a • n may not be even numbered • Alter final iteration of loop • n > 3 not guaranteed • Special case for n = 1, 2, 3 • Is not optimal for all inputs, e.g. 1, 20, 21, 22 • Can you quantify the nature of these inputs? Suggest an alternative. • Final solution is left as an exercise. Attempt to make your solution elegant

  13. Fundamentals of Algorithmic Problem Solving • Understanding the Problem • Make sure you are solving the correct problem and for all legitimate inputs • Ascertaining the Capabilities of a Computational Device • Sequential vs. Parallel. • What are the speed and memory limits? • Choosing between exact and approximate Problem Solving • Is absolute precision required? Sometimes this may not be possible • Deciding on Appropriate Data Structures • Algorithms often rely on carefully structuring the data • Fundamental Data Structures: array, linked list, stacks, queues, heaps, graphs, trees, sets

  14. Fundamentals of Algorithm Design • Applying an Algorithm Design Technique • Using a general approach to problem solving that is applicable to a variety of problems • Specifying the Algorithm • Pseudocode is a mixture of natural language and programming constructs that has replaced flowcharts • Proving an Algorithms Correctness • Prove that an algorithm yields a required result for legitimate inputs in finite time • Analyzing an Algorithm • Consider time efficiency, space efficiency, simplicity, generality, optimality • Analysis can be empirical or theoretical • Coding an Algorithm

  15. Well known Computational Problems • Sorting • Searching • String Processing • String Matching • Graph Problems • Graph Traversal, Shortest Path, Graph Colouring • Combinatorial Problems • Find a combinatorial object - permutation, combination, subset - subject to constraints • Geometric Problems • Closest-Pair, Convex-Hull • Numerical Problems • Solving systems of equations, computing definite integrals, evaluating functions, etc.

  16. Algorithm Design Strategies • Brute force • A straightforward approach to solving a problem, usually directly based on the problem’s statement • Divide and conquer • Divide a problem into smaller instances, solve smaller instances (perhaps recursively), combine • Decrease and conquer • Exploit relationship between the problem and a smaller instance reduced by some factor (often 1) • Transform and conquer • Transform the problem to a simpler instance, another representation or an instance with a known solution

  17. More Algorithm Design Strategies • Greedy approach • Make locally optimal steps which (hopefully) lead to a globally optimal solution for an optimization problem • Dynamic programming • Technique for solving problems with overlapping sub-domains • Backtracking and Branch and bound • A way of tackling difficult optimization and combinatorial problems without exploring all state-space • Space and time tradeoffs • Preprocess the input and store additional information to accelerate solving the problem

  18. How to Solve It: Understanding the Problem • Taken from G. Polya, “How to Solve It”, 2nd edition. A classic textbook on problem solving for mathematics • You have to understand the problem. • What is the unknown? What are the data? Is the problem statement sufficient, redundant, contradictory • Draw a figure. Introduce suitable notation • Separate the various parts of the problem. Can you write them down?

  19. Devising a Plan • Find the connection between the data and the unknown. You may be obliged to consider auxiliary problems if an immediate connection cannot be found. You should obtain eventually a plan of the solution. • Have you seen it before? Or have you seen the same problem in a slightly different form? • Do you know a related problem? Do you know a theorem that could be useful? • Look at the unknown! And try to think of a familiar problem having the same or a similar unknown. • Could you restate the problem? Could you restate it still differently? Go back to definitions. • If you cannot solve the proposed problem try to solve first some related problem. Are the unknown and the new data nearer to each other? • Did you use all the data? Did you use the whole condition? Have you taken into account all essential notions involved in the problem?

  20. Carrying it Through • Carry out the Plan • Carrying out your plan of the solution, check each step. Can you see clearly that the step is correct? Can you prove that it is correct? • Looking Back • Can you check the result? Can you check the argument? • Can you derive the solution differently? Can you see it at a glance? • Can you use the result, or the method, for some other problem?

  21. Analysis of Algorithms • How good is the algorithm? • Correctness • Time efficiency • Space efficiency • Simplicity • Does there exist a better algorithm? • Lower bounds • Optimality

  22. Why Study Algorithms? • Theoretical importance • The core of computer science • Practical importance • A practitioner’s toolkit of known algorithms • Framework for designing and analyzing algorithms for new problems • Useful mindset

  23. Course Structure • Fundamentals of the Analysis of Algorithms (Ch. 2) • Asymptotic notations, analysis of recursive and non-recursive algorithms, empirical analysis • Algorithmic Strategies (Ch. 3-9) • Brute force, Divide-and-Conquer, Decrease-and-Conquer, Transform-and-Conquer, Space and Time Tradeoffs, Greedy Techniques, Biologically-inspired techniques, Dynamic Programming • Limitations of Algorithms (Ch. 10 + handouts) • Turing Machines, Computability, Problem Classification • Coping with Limitations on Algorithms (Ch. 11) • Backtracking and Branch and Bound • Anany Levitin, “Introduction to the Design and Analysis of Algorithms”, International Edition, Addison-Wesley, 2003

  24. Practicals • Weekly mini prac exams • Given a problem specification that is solvable using the algorithm design strategies presented in the course • Design Algorithm • Code it in C++ • Submit it for automatic marking • After the 3-hour lab session will be asked to do a short analysis of the solution

More Related